Joint Distributions, Covariance and Independence: every key term you need (+ practice quiz)
25 flashcard terms for Probability & Statistical Inference Topic 4, written to match the course framework. Study them here, then drill them as interactive flashcards, or test yourself with the 15-question quiz — free, no account needed.
The rule assigning probability to combinations of values of two or more variables at once. It carries strictly more information than the separate distributions, since it also encodes how the variables move together.
Joint mass function
A function giving the probability of each pair of values for two discrete variables, nonnegative and summing to one over the grid of possibilities. Row and column sums recover the individual distributions.
Joint density function
A nonnegative surface over the plane whose volume above a region gives the probability of landing there. Total volume is one, and probabilities for single curves or points are zero.
Marginal distribution
The distribution of one variable obtained by summing or integrating the joint object over all values of the others. Marginalising discards dependence information and cannot be reversed without extra assumptions.
Conditional distribution
The distribution of one variable once another is fixed at a value, formed by dividing the joint object by the corresponding marginal. It is a genuine distribution in its own right and integrates to one.
Independence of random variables
The condition that the joint mass or density factors into the product of the marginals for every pair of values. Equivalently, the conditional distribution of one variable does not depend on the other.
Product support requirement
Independent variables must have a joint support that is a rectangle in the sense that the range of one does not depend on the value of the other. A triangular support alone already rules out independence.
Covariance
The expected product of the deviations of two variables from their means, positive when they tend to exceed their means together. Its size depends on the units of both variables, which limits its interpretability.
Correlation coefficient
Covariance divided by the product of the standard deviations, a unit-free number between minus one and one. It measures linear association only and attains the extremes exactly under an exact linear relation.
Uncorrelated but dependent
Zero correlation means no linear association, not independence. A variable and its own square on a symmetric range are a standard example: strongly dependent yet exactly uncorrelated.
Bilinearity of covariance
Covariance is linear in each argument separately, so covariances of sums expand term by term with constants pulled out. This is the machinery behind variance formulas for weighted sums.
Variance of a sum
The variance of a sum equals the sum of the variances plus twice the sum of all pairwise covariances. Under independence the cross terms vanish, which is why independence is assumed so often.
Covariance matrix
A symmetric array holding variances on its diagonal and covariances off it. It is positive semidefinite, so no vector of weights can ever produce a negative variance for a linear combination.
Bivariate normal distribution
A joint model determined by two means, two variances and one correlation. It is the rare family where zero correlation genuinely does imply independence, a fact that does not extend to other joint models.
Conditional mean function
The mean of one variable as a function of the value of another. For the bivariate normal it is exactly linear, which is the theoretical justification for straight-line regression in that setting.
Conditional variance in the normal case
For the bivariate normal, the spread of one variable given the other does not depend on the conditioning value and shrinks by a factor of one minus the squared correlation. This is homoscedasticity in its cleanest form.
Convolution of distributions
The operation producing the distribution of a sum of independent variables by summing or integrating over all ways the parts can add to a total. Generating functions usually make this far easier.
Closure under addition
Independent normal variables add to a normal variable, independent Poisson counts add to a Poisson count, and gamma variables with a shared rate add to a gamma variable. These closures follow from multiplying generating functions.
Order statistics
The sorted values of a sample, from smallest to largest. Their distributions follow from the cumulative distribution function of the parent and underlie results about sample extremes, medians and ranges.
Exchangeability
The property that the joint distribution is unchanged under any reordering of the variables. It is weaker than independence yet strong enough to support many symmetry arguments in sampling.
Copula
A joint distribution on the unit square with uniform margins, used to separate the dependence structure from the individual marginal shapes. It shows that identical correlations can hide very different tail behaviour.
Simpson's paradox
An association present in every subgroup can reverse when the groups are pooled, because the grouping variable is related to both others. It is a warning that marginal and conditional relationships can point opposite ways.
Regression to the mean
When two variables are imperfectly correlated, extreme values of one are paired on average with less extreme values of the other. It is a statistical consequence of imperfect correlation, not evidence of any causal effect.
Multivariate transformation
Deriving a joint density for functions of several variables using the absolute determinant of the Jacobian matrix of the inverse map. Careful tracking of the transformed support is as important as the determinant itself.
Random sample as a joint model
A random sample is a collection of independent variables with a common distribution, so its joint density is the product of identical factors. That product form is exactly what later becomes the likelihood function.