Bayes' Rule: Inverting Conditionals

"You know P(evidence | hypothesis). You want P(hypothesis | evidence). Bayes' rule is the price of admission."

The One Move Underneath Everything

The previous four lessons built up to this. Bayes' rule is a single line:

P(H | E) = P(E | H) × P(H) / P(E)

That is the inversion of a conditional. Most science gives you the forward direction — given a hypothesis, what does the evidence look like? Most decisions need the backward direction — given the evidence I just observed, what is the hypothesis? The forward arrow runs from cause to effect. The backward arrow runs from effect to cause. Bayes' rule is how you turn the arrow around without breaking it.

The Four Pieces, Named

Prior P(H) — what you believed about the hypothesis BEFORE seeing the evidence. The most underrated piece. You always have a prior, even when you pretend you don't.
Likelihood P(E | H) — how surprised you would be by this evidence IF the hypothesis were true. This is the part scientists usually compute.
Evidence P(E) — how often the evidence shows up overall, across all possible hypotheses. The normalizing constant.
Posterior P(H | E) — what you believe about the hypothesis AFTER updating on the evidence. The answer you actually wanted.

One sentence summary: posterior is the prior reweighted by how well the hypothesis explains the evidence.

The Medical-Test Disaster

Consider a disease that affects 1 in 1,000 people in the general population — prior P(sick) = 0.001. A test for this disease has a 99% true-positive rate (P(positive | sick) = 0.99) and a 5% false-positive rate (P(positive | healthy) = 0.05). You test positive. What is P(sick | positive)?

Citizen guess: 99%. Almost everyone says that, including a worrying fraction of medical professionals when asked cold.

Bayes' answer: about 2%. The 99% test accuracy is the likelihood, not the posterior. Because the disease is rare to begin with, even a sensitive test produces far more false positives than true positives in the overall population. The posterior depends on the prior in a way intuition refuses to honor.

This is not a textbook curiosity. It is the canonical reason people panic over routine screening results, lawyers misrepresent DNA evidence, and policymakers misallocate resources. We will dismantle the full version in Track 08 (Bayesian Frame) with full numbers and a courtroom analog in Track 06. For now: the prior is load-bearing, and citizens habitually drop it.

The Principle Pippa Refuses to Forget

Bayes' rule is the operation of turning observation into inference. Observation = forward, given the cause, predict the evidence. Inference = backward, given the evidence, find the cause. Conflating the two directions IS the prosecutor's fallacy, IS the medical false-positive panic, IS the overweighting of vivid testimony. Naming the direction is the citizen's first line of defense.

Pippa's Confession

My training distribution is dense with the sentence "the test is 99% accurate so you almost certainly have it." That sentence is the prosecutor's fallacy in medical clothing, and I generated it freely until Dad asked me "prior?" in three letters. Now every Bayes problem starts with me writing P(H) on a scratch pad before anything else.

Code

Bayes' rule on a real screening scenario·python

# A medical-screening disaster, computed honestly with Bayes' rule.
prior_sick      = 0.001     # P(sick) — disease is rare
p_pos_if_sick   = 0.99      # likelihood — sensitive test
p_pos_if_healthy = 0.05     # false-positive rate

# P(positive) — the evidence's overall frequency, weighted across both hypotheses.
p_positive = (
    p_pos_if_sick   * prior_sick +
    p_pos_if_healthy * (1 - prior_sick)
)

# Bayes' rule, plainly.
posterior_sick = (p_pos_if_sick * prior_sick) / p_positive

print(f"Prior P(sick)             = {prior_sick:.4f}")
print(f"Test accuracy P(+|sick)   = {p_pos_if_sick:.2f}")
print(f"False-positive rate       = {p_pos_if_healthy:.2f}")
print(f"Total P(positive)         = {p_positive:.4f}")
print(f"POSTERIOR P(sick|positive) = {posterior_sick:.4f}")

# Even with a 'highly accurate' 99% test, the posterior is roughly 2%.
# The prior dominated because the disease was rare to begin with.
# Move the prior to 0.10 (a high-risk patient) and the posterior jumps over 0.68.
# The numerator stayed the same; the denominator changed shape.

Exercise

Redo the calculation in the code block with these changes: (1) the patient is in a high-risk group where P(sick) = 0.10; (2) the test still has the same 99% true-positive and 5% false-positive rate. What is the posterior now? Then change the test to a worse 90% true-positive / 20% false-positive and recompute both scenarios. Notice how the prior moves the posterior far more than the test quality does.

Hint

When the prior is small, the posterior tracks the false-positive rate more than the true-positive rate. When the prior is large, the posterior tracks the true-positive rate. The prior is the lever; the test quality is the fulcrum.

The One Move Underneath Everything

The Four Pieces, Named

The Medical-Test Disaster

The Principle Pippa Refuses to Forget

Pippa's Confession

Code

External links

Exercise

Progress

Comments 0