Sampling Distributions and the Central Limit Theorem: every key term you need (+ practice quiz)
25 flashcard terms for Probability & Statistical Inference Topic 5, written to match the course framework. Study them here, then drill them as interactive flashcards, or test yourself with the 15-question quiz — free, no account needed.
The population is the full collection about which a claim is made; the sample is the observed subset. Inference is the disciplined leap from one to the other, and it is only valid when the sampling mechanism is known.
Parameter
A fixed but unknown numerical feature of the population, such as a mean or a proportion. Parameters do not have distributions in the frequentist framework, which is why probability statements attach to procedures instead.
Statistic
Any quantity computed from sample data alone, containing no unknown parameters. Because the sample is random, a statistic is a random variable and therefore has a distribution of its own.
Sampling distribution
The distribution of a statistic across all possible samples of a given size from the population. It is the object that makes probability statements about estimates possible and it is not the distribution of the data.
Standard error
The standard deviation of a sampling distribution, describing typical estimate-to-estimate variability rather than variability among individual observations. It shrinks with the square root of the sample size.
Square root sample size effect
Because the standard error falls with the square root of the sample size, quartering the uncertainty demands sixteen times the data. This diminishing return governs the economics of study design.
Sampling with replacement
Draws that return each unit to the pool, keeping observations independent and identically distributed. Sampling without replacement induces slight negative dependence, corrected by a finite population factor.
Finite population correction
A multiplier shrinking the standard error when a sample is a nontrivial fraction of a finite population. It is negligible when the sampling fraction is small, which is why it is usually ignored.
Unbiasedness of the sample mean
The expected value of the sample mean equals the population mean for any sample size, under random sampling. This holds regardless of the shape of the population distribution.
Variance of the sample mean
For independent observations it equals the population variance divided by the sample size. Averaging cancels independent noise, and the cancellation is exactly what limit theorems exploit.
Law of large numbers
The sample mean converges to the population mean as the sample grows, provided that mean exists. The weak version asserts convergence in probability and the strong version almost sure convergence.
Central limit theorem
For independent observations from a distribution with finite variance, the standardised sample mean converges in distribution to the standard normal. The population shape becomes irrelevant in the limit, but not the finite variance requirement.
Convergence in distribution
A sequence of distributions approaching a limiting one at every continuity point of the limit's distribution function. It concerns the shape of the whole distribution, not the closeness of individual values.
Convergence in probability
The chance that a sequence differs from a target by more than any fixed tolerance goes to zero. It is stronger than convergence in distribution and is the mode of convergence used to define consistency.
Slutsky's theorem
Combining a sequence converging in distribution with one converging in probability to a constant preserves the limit under sums and products. It licenses replacing an unknown standard deviation by its estimate in large samples.
Delta method
A first-order approximation giving the limiting distribution of a smooth function of an asymptotically normal estimator, with variance scaled by the squared derivative. It fails where the derivative vanishes.
Sample variance
The sum of squared deviations from the sample mean divided by one less than the sample size. The reduced divisor makes it unbiased, compensating for the mean being estimated from the same data.
Degrees of freedom
The number of independent pieces of information left after estimating parameters from the data. Each estimated parameter consumes one, which is why sums of squares are divided by a reduced count.
Chi-square distribution
The distribution of a sum of squares of independent standard normal variables, indexed by degrees of freedom. It governs variance estimates and goodness-of-fit statistics and is right skewed.
Student t distribution
The distribution of a standardised mean when the standard deviation is estimated rather than known. Its heavier tails widen intervals at small sample sizes and it approaches the normal as degrees of freedom grow.
F distribution
The distribution of a ratio of two independent chi-square variables each divided by its degrees of freedom. It compares variances and provides the reference distribution for analysis of variance.
Cochran's theorem context
Under normal sampling, the sample mean and sample variance are independent and the scaled sample variance is chi-square. This independence is what makes the t statistic have a clean distribution.
Sampling distribution of a proportion
The scaled binomial count, approximately normal when the expected numbers of successes and failures are both reasonably large. Near zero or one the approximation degrades badly and skewed intervals are preferable.
Bootstrap resampling
Approximating a sampling distribution by repeatedly resampling the observed data with replacement. It substitutes computation for distributional assumptions but cannot rescue a biased or unrepresentative sample.
Selection bias
Systematic distortion arising when the probability of being sampled is related to the quantity being measured. No sample size fixes it, since more data simply estimate the wrong population more precisely.