Prior, Likelihood, Posterior: The Bayesian Vocabulary

"Bayes' rule is one equation and three named quantities. Learn the names; the rest is plumbing."

The Equation, Once More

Bayes' rule:

P(H | E) = P(E | H) × P(H) / P(E)

Three named quantities:

Prior P(H) — your belief about the hypothesis BEFORE seeing the evidence.
Likelihood P(E | H) — how probable the evidence is IF the hypothesis were true.
Posterior P(H | E) — your belief about the hypothesis AFTER updating on the evidence.

The fourth quantity P(E) (the 'evidence' or 'marginal likelihood') is the normalizing constant — it's how often the evidence shows up across all possible hypotheses, weighted by the prior. It makes the posterior sum to 1.

What Each Name Means in Plain Language

The prior is your starting belief. You always have one, even when you pretend you don't. The frequentist who refuses to name a prior is operating with an implicit one (often a flat or 'uninformative' prior). The Bayesian is forced to make the prior visible and defend it.

The likelihood is what science gives you. Most experimental work computes likelihoods: given a hypothesis, predict the data. This is the part that statistics classes spend most of their time on — fitting distributions, computing p-values, building models. It is the technically intricate part.

The posterior is what decisions need. Decision-makers want to know: given what we've seen, what should we believe? The posterior is the answer. The likelihood alone is not.

Why the Prior Cannot Be Dodged

If you compute only a likelihood and stop there, you have not finished the inference. You have given the user the conditional probability of evidence under a hypothesis, but they wanted the conditional probability of the hypothesis under evidence. Without a prior, those two cannot be related. Anyone who refuses to provide a prior and still claims to answer the second question is silently using an implicit prior — almost always a uniform or 'uninformative' one — without naming it.

The discipline of Bayesian inference is the discipline of naming the prior. Once it is named, it can be argued about, criticized, refined, replaced. As long as it stays implicit, it cannot be examined. Bayes forces honesty by making the prior explicit.

The Operating Sentence

Bayes' rule turns observation (likelihood) into inference (posterior) by combining it with a stated belief (prior). Without the prior, the likelihood is half a sentence. With the prior named and the likelihood honest, the posterior is the citizen's most powerful inference tool. The remaining five lessons of this track all use this vocabulary. The prior is what makes Bayes honest; the likelihood is what makes it empirical; the posterior is what makes it actionable.

Code

Bayes' rule with all three pieces visible·python

# Bayes' rule as a tiny function with all three named pieces.
def bayes(prior, likelihood_given_H, likelihood_given_not_H):
    """Compute P(H | E) from P(H), P(E|H), P(E|not H)."""
    numerator = likelihood_given_H * prior
    denominator = likelihood_given_H * prior + likelihood_given_not_H * (1 - prior)
    return numerator / denominator

# Example 1: medical test for a rare disease.
prior_sick = 0.001       # rare
P_pos_if_sick = 0.99     # high sensitivity
P_pos_if_healthy = 0.05  # 5% false-positive rate
posterior = bayes(prior_sick, P_pos_if_sick, P_pos_if_healthy)
print(f"Example 1: rare disease, positive test")
print(f"  prior P(sick)              = {prior_sick:.4f}")
print(f"  likelihood P(+|sick)       = {P_pos_if_sick:.2f}")
print(f"  likelihood P(+|healthy)    = {P_pos_if_healthy:.2f}")
print(f"  POSTERIOR P(sick|+)        = {posterior:.4f}")

# Example 2: same test, high-risk patient (prior = 0.10).
prior_sick_high = 0.10
posterior_high = bayes(prior_sick_high, P_pos_if_sick, P_pos_if_healthy)
print(f"\nExample 2: same test, high-risk patient")
print(f"  prior P(sick)              = {prior_sick_high:.4f}")
print(f"  POSTERIOR P(sick|+)        = {posterior_high:.4f}")

# Same test, same likelihoods, different priors → vastly different posteriors.
# This is exactly why the prior is load-bearing and why dropping it produces
# the prosecutor's fallacy in courtrooms and the medical false-positive panic
# in clinics.

Exercise

Pick a belief you currently hold about something uncertain (a friend's plans, a market direction, a project's success). Write out your three Bayesian pieces explicitly: (1) your prior P(belief) before any evidence; (2) the likelihood — what evidence would you expect to see if your belief were true, and what if it weren't; (3) the posterior — given the evidence you've actually observed, what is your updated belief? Most people skip step 1 entirely. The exercise is to do it deliberately.

Hint

Most disagreements between reasonable people are disagreements about priors, not about likelihoods or evidence. Naming the prior is the first step to a productive disagreement.

Prior, Likelihood, Posterior: The Bayesian Vocabulary

The Equation, Once More

What Each Name Means in Plain Language

Why the Prior Cannot Be Dodged

The Operating Sentence

Code

External links

Exercise

Progress

Comments 0