Probability Axioms, Conditional Probability and Bayes' Theorem: every key term you need (+ practice quiz)
25 flashcard terms for Probability & Statistical Inference Topic 1, written to match the course framework. Study them here, then drill them as interactive flashcards, or test yourself with the 15-question quiz — free, no account needed.
The set of all outcomes a random experiment can produce, usually written S or Omega. Every event is a subset of it, so choosing the sample space is the first modelling decision and it fixes what questions can even be asked.
Event
Any subset of the sample space, including the empty set and the whole space. In continuous settings only measurable subsets qualify, which is why probability is defined on a sigma-algebra rather than on every conceivable subset.
Sigma-algebra
A collection of subsets of the sample space that contains the whole space and is closed under complement and countable union. It is the domain of the probability function and guarantees that limits of events remain events.
Kolmogorov axioms
Probability is nonnegative, assigns one to the whole sample space, and is countably additive over disjoint events. Every familiar rule, including complements and inclusion-exclusion, is derived from just these three requirements.
Countable additivity
For a sequence of pairwise disjoint events, the probability of their union equals the sum of their probabilities. This stronger-than-finite requirement is what makes continuity of probability along monotone sequences of events work.
Complement rule
The probability that an event fails equals one minus the probability that it happens. It is the workhorse behind at-least-one problems, where computing the complement of no successes is far easier than direct enumeration.
Inclusion-exclusion principle
The probability of a union adds the individual probabilities, subtracts all pairwise intersections, adds back triples, and so on. Skipping the correction terms double counts outcomes lying in more than one of the events.
Equally likely outcomes model
A finite sample space in which every outcome carries the same probability, so probability reduces to counting favourable cases over total cases. It is a modelling assumption, not a law, and it fails whenever outcomes are unevenly weighted.
Permutation count
The number of ordered arrangements of k items drawn from n distinct items, equal to n factorial divided by the factorial of n minus k. Order matters here, which distinguishes it from a combination.
Combination count
The number of unordered subsets of size k from n distinct items, the binomial coefficient n choose k. Used whenever the identity of the chosen group matters but the sequence of selection does not.
Conditional probability
The probability of A given B equals the probability of their intersection divided by the probability of B, defined only when B has positive probability. Conditioning renormalises the sample space to the outcomes consistent with B.
Multiplication rule
The probability of an intersection equals the probability of one event times the conditional probability of the other given it. Chaining this along a sequence gives the probability of any path through a tree diagram.
Independent events
Two events are independent when the probability of their intersection equals the product of their probabilities, equivalently when conditioning on one leaves the other unchanged. Independence is an assumption about the model, not something visible in a Venn diagram.
Mutually exclusive events
Events that cannot both occur, so their intersection is empty. Two exclusive events with positive probability are necessarily dependent, since knowing one occurred drives the other to probability zero, a point often confused with independence.
Pairwise versus mutual independence
A family can be independent in every pair yet fail joint independence, because the product rule must hold for every sub-collection, not just pairs. Mutual independence is strictly stronger and is what most theorems require.
Partition of the sample space
A collection of disjoint events whose union is the whole space, so exactly one of them occurs. Partitions are the scaffolding for the law of total probability and for case-splitting arguments generally.
Law of total probability
The probability of an event equals the sum over a partition of the conditional probabilities weighted by the partition probabilities. It converts an awkward unconditional question into a set of easier conditional ones.
Bayes' theorem
The posterior probability of a hypothesis given evidence is proportional to the likelihood of that evidence times the prior probability of the hypothesis, normalised by the total probability of the evidence. It reverses the direction of conditioning.
Prior probability
The probability assigned to a hypothesis before the current evidence is taken into account. Its value materially changes the posterior, which is why rare-condition screening results stay unconvincing even with an accurate test.
Posterior probability
The updated probability of a hypothesis after conditioning on observed evidence. Repeated updating is coherent: yesterday's posterior serves as today's prior when a further independent observation arrives.
Likelihood in Bayes updating
The probability of the observed evidence computed under each competing hypothesis. Only ratios of likelihoods matter for the update, since the common normalising denominator cancels when comparing two hypotheses.
Base rate fallacy
The error of judging a positive test result by its accuracy alone while ignoring how rare the condition is. When prevalence is very low, most positives are false positives even for a highly specific test.
Prosecutor's fallacy
Confusing the probability of the evidence given innocence with the probability of innocence given the evidence. The two conditionals differ unless the prior odds happen to be even, which is exactly what is in dispute.
Odds form of Bayes
Posterior odds equal prior odds times the likelihood ratio. This version strips away the normalising constant and makes clear that evidence multiplies belief rather than setting it to an absolute level.
Conditional independence
Two events independent once a third is conditioned on, though possibly dependent unconditionally. This is the assumption behind naive Bayes classifiers and it can fail badly when the conditioning variable omits a shared cause.