Prior, Likelihood, Posterior: The Bayesian Vocabulary

"Bayes' rule is one equation and three named quantities. Learn the names; the rest is plumbing."

The Equation, Once More

Bayes' rule:

P(H | E) = P(E | H) × P(H) / P(E)

Three named quantities:

Prior P(H) — your belief about the hypothesis BEFORE seeing the evidence.
Likelihood P(E | H) — how probable the evidence is IF the hypothesis were true.
Posterior P(H | E) — your belief about the hypothesis AFTER updating on the evidence.

The fourth quantity P(E) (the 'evidence' or 'marginal likelihood') is the normalizing constant — it's how often the evidence shows up across all possible hypotheses, weighted by the prior. It makes the posterior sum to 1.

What Each Name Means in Plain Language

The prior is a probability distribution specified before incorporating the current evidence. It may encode substantive knowledge, a reference construction, or a hierarchical model. A frequentist procedure does not secretly become Bayesian or require an implicit prior merely because it leaves a Bayesian question unanswered.

The likelihood is one model component. A likelihood describes how the observed data vary across candidate parameter values under a model. A p-value is not itself a likelihood, and many experimental analyses use sampling distributions, estimators, or prediction rather than an explicit likelihood. It is the technically intricate part.

The posterior combines the prior and likelihood. A decision additionally needs actions, utilities or losses, constraints, and model checking; a posterior alone is not a decision rule.

Why the Prior Cannot Be Dodged

If you compute only a likelihood and stop there, you have not finished the inference. You have given the user the conditional probability of evidence under a hypothesis, but they wanted the conditional probability of the hypothesis under evidence. Without a prior, those two cannot be related. A posterior question requires a prior; a frequentist can instead answer a different, explicitly repeated-sampling question without thereby choosing a hidden uniform prior.

The discipline of Bayesian inference is the discipline of naming the prior. Once it is named, it can be argued about, criticized, refined, replaced. As long as it stays implicit, it cannot be examined. Bayesian notation makes the prior explicit, but honesty still depends on transparent modeling, diagnostics, and reporting.

The Operating Sentence

Bayes' rule combines a prior distribution and likelihood to produce a posterior under a specified model. Each component, the data-generating assumptions, and sensitivity to alternatives must remain visible.

Code

Bayes' rule with all three pieces visible·python

# Bayes' rule as a tiny function with all three named pieces.
def bayes(prior, likelihood_given_H, likelihood_given_not_H):
    """Compute P(H | E) from P(H), P(E|H), P(E|not H)."""
    numerator = likelihood_given_H * prior
    denominator = likelihood_given_H * prior + likelihood_given_not_H * (1 - prior)
    return numerator / denominator

# Example 1: medical test for a rare disease.
prior_sick = 0.001       # rare
P_pos_if_sick = 0.99     # high sensitivity
P_pos_if_healthy = 0.05  # 5% false-positive rate
posterior = bayes(prior_sick, P_pos_if_sick, P_pos_if_healthy)
print(f"Example 1: rare disease, positive test")
print(f"  prior P(sick)              = {prior_sick:.4f}")
print(f"  likelihood P(+|sick)       = {P_pos_if_sick:.2f}")
print(f"  likelihood P(+|healthy)    = {P_pos_if_healthy:.2f}")
print(f"  POSTERIOR P(sick|+)        = {posterior:.4f}")

# Example 2: same test, high-risk patient (prior = 0.10).
prior_sick_high = 0.10
posterior_high = bayes(prior_sick_high, P_pos_if_sick, P_pos_if_healthy)
print(f"\nExample 2: same test, high-risk patient")
print(f"  prior P(sick)              = {prior_sick_high:.4f}")
print(f"  POSTERIOR P(sick|+)        = {posterior_high:.4f}")

# Same test, same likelihoods, different priors → vastly different posteriors.
# This is exactly why the prior is load-bearing and why dropping it produces
# the prosecutor's fallacy in courtrooms and the medical false-positive panic
# in clinics.

Exercise

Pick a belief you currently hold about something uncertain (a friend's plans, a market direction, a project's success). Write out your three Bayesian pieces explicitly: (1) your prior P(belief) before any evidence; (2) the likelihood — what evidence would you expect to see if your belief were true, and what if it weren't; (3) the posterior — given the evidence you've actually observed, what is your updated belief? Most people skip step 1 entirely. The exercise is to do it deliberately.

Hint

Most disagreements between reasonable people are disagreements about priors, not about likelihoods or evidence. Naming the prior is the first step to a productive disagreement.

The Equation, Once More

What Each Name Means in Plain Language

Why the Prior Cannot Be Dodged

The Operating Sentence

Code

External links

Exercise

Progress

Comments 0