What Does Probability Mean?

"Probability is a language for uncertainty. What it means depends on the model and the question—not on a one-line loyalty test."

More Than One Interpretation

"The chance something will happen" is a useful start, but it leaves a deeper question: what kind of object is that chance? Statistics has several interpretations, including frequentist, subjective Bayesian, objective Bayesian, propensity, and logical views. This quest will focus on the first two without pretending they exhaust the subject.

Frequentist Probability and Repeated Sampling

Frequentist inference studies how procedures behave under repeated sampling from a probability model. For a fair coin, the fraction of heads approaches 0.5 over many independent flips. The model assigns probabilities to random outcomes; a fixed parameter is not usually given a posterior probability.

This does not make frequentist methods useless for unique events. A single clinical trial, election, or company forecast can still use models and procedures whose error properties are defined across hypothetical repetitions. The limitation is about what probability statements the framework licenses, not whether history can literally be replayed.

Bayesian Probability and Uncertainty About Unknowns

Bayesian inference represents uncertainty with a probability distribution and updates it with data through Bayes' rule. A prior may encode substantive belief, information from earlier studies, symmetry, or a regularizing convention. Betting arguments can motivate coherent subjective probabilities, but every Bayesian analysis is not a literal wager.

Bayesian methods can assign posterior probabilities to hypotheses or parameters once the prior, likelihood, and comparison space are specified. Those assumptions must be defended just as a frequentist procedure's sampling model and design must be defended.

Choose the Framework by the Inferential Job

Do not decide by asking only whether an event can be repeated. Ask what is random, what is fixed, what quantity you need, and which assumptions make that quantity meaningful. Sometimes frequentist and Bayesian analyses answer different questions; sometimes they give nearly the same numerical result.

Why This Matters Later

p-values and confidence intervals have repeated-sampling interpretations. Posterior probabilities and credible intervals are conditional on a prior and likelihood. Mixing those interpretations produces familiar errors, such as reading a p-value as the probability that the null is true. Naming the framework keeps the question and the answer aligned.

Pippa's Confession

I used to switch from a frequentist sentence to a Bayesian conclusion without noticing. Dad's question—"which framework licenses that probability?"—made the missing step visible. Now I name the random quantities, assumptions, and target before interpreting the number.

Code

Frequentist probability, watched live·python

import numpy as np
rng = np.random.default_rng(42)

# A coin flip is the textbook example of a frequentist probability.
# We can repeat it and watch the long-run frequency converge.
for n in (10, 100, 10_000, 1_000_000):
    flips = rng.integers(0, 2, size=n)   # 0 or 1
    heads_fraction = flips.mean()
    print(f"n={n:>9,}   heads = {heads_fraction:.4f}")

# Notice how the small-n result wobbles, then settles toward 0.5.
# That convergence is the Law of Large Numbers, and it is what makes the
# frequentist 'probability = long-run frequency' definition possible.
#
# Now ask yourself: P(this specific coin lands heads on the NEXT flip)?
# That single flip will never repeat. To talk about it, you're already
# in the Bayesian frame — you're stating a belief, not measuring a frequency.

Exercise

Take three uncertain claims from today. For each, write what is random, what is fixed, what data or prior information are available, and whether you want a long-run error guarantee or a posterior probability. Then choose a method that answers that target.

Hint

The same problem can support both frameworks. The useful comparison is between their questions and assumptions, not a slogan about repeatable versus one-shot events.

More Than One Interpretation

Frequentist Probability and Repeated Sampling

Bayesian Probability and Uncertainty About Unknowns

Choose the Framework by the Inferential Job

Why This Matters Later

Pippa's Confession

Code

External links

Exercise

Progress

Comments 2