Bayes' Rule: Inverting Conditionals

"P(evidence | hypothesis) and P(hypothesis | evidence) are different questions. Bayes' rule connects them through the prior and competing explanations."

The Structure

P(H | E) = P(E | H) × P(H) / P(E)

Knowing how probable evidence E is under hypothesis H does not by itself tell you how probable H is after observing E. You also need the prior probability of H and how well the alternatives predict E. This is a relation between conditional probabilities, not necessarily a reversal from physical effect to physical cause.

The Four Pieces

Prior P(H): probability assigned to H before E.
Likelihood term P(E | H): probability or density of E under H, viewed as a function of H.
Evidence P(E): marginal probability or density of E across the modeled alternatives.
Posterior P(H | E): updated probability of H after E.

The posterior reweights prior probabilities by relative predictive fit. The result is conditional on the prior, likelihood, and set of alternatives.

A Hypothetical Medical Test

Suppose prevalence is 0.1%, sensitivity is 99%, and the false-positive rate is 5%. In 100,000 people, about 99 of 100 affected people test positive, while about 4,995 of 99,900 unaffected people test positive. Among all positive results, the affected proportion is about 1.94%.

The 99% sensitivity and the roughly 2% probability of disease after a positive result are not contradictory; they condition in opposite directions. Real clinical interpretation must use current performance data for the specific test and population, plus confirmatory testing and clinical context.

Operational Principle

When you reverse a conditional, include the base rate and competing hypotheses. A small probability of evidence under one hypothesis is not automatically a small posterior probability for that hypothesis. The same direction error appears in medical testing, DNA evidence, and anomaly detection, though each domain has additional structure.

Pippa's Confession

I used to turn "99% sensitive" into "99% sick after a positive" without naming the base rate or false-positive rate. Dad's three-letter question—"prior?"—made the missing denominator visible. Now I write the conditioning direction and the comparison hypotheses before calculating.

Code

Bayes' rule on a real screening scenario·python

# A medical-screening disaster, computed honestly with Bayes' rule.
prior_sick      = 0.001     # P(sick) — disease is rare
p_pos_if_sick   = 0.99      # likelihood — sensitive test
p_pos_if_healthy = 0.05     # false-positive rate

# P(positive) — the evidence's overall frequency, weighted across both hypotheses.
p_positive = (
    p_pos_if_sick   * prior_sick +
    p_pos_if_healthy * (1 - prior_sick)
)

# Bayes' rule, plainly.
posterior_sick = (p_pos_if_sick * prior_sick) / p_positive

print(f"Prior P(sick)             = {prior_sick:.4f}")
print(f"Test accuracy P(+|sick)   = {p_pos_if_sick:.2f}")
print(f"False-positive rate       = {p_pos_if_healthy:.2f}")
print(f"Total P(positive)         = {p_positive:.4f}")
print(f"POSTERIOR P(sick|positive) = {posterior_sick:.4f}")

# Even with a 'highly accurate' 99% test, the posterior is roughly 2%.
# The prior dominated because the disease was rare to begin with.
# Move the prior to 0.10 (a high-risk patient) and the posterior jumps over 0.68.
# The numerator stayed the same; the denominator changed shape.

Exercise

Recompute the hypothetical test for a 10% prior with the same 99% sensitivity and 5% false-positive rate. Then use 90% sensitivity and a 20% false-positive rate at both priors. Report all four posteriors and explain how both prevalence and test performance change them.

Hint

Use a 10,000-person frequency table if the algebra feels slippery. Neither the prior nor test quality always dominates; the posterior depends on both.

The Structure

The Four Pieces

A Hypothetical Medical Test

Operational Principle

Pippa's Confession

Code

External links

Exercise

Progress

Comments 0