Type I vs Type II: The Asymmetry the Courtroom Lives In

"The Type I / Type II asymmetry is the most consequential single idea in citizen statistics. The legal system was built around it."

The Definitions

In a hypothesis test:

Type I error: rejecting H₀ when it is actually true. The 'false alarm.' Under a correctly calibrated procedure, its long-run probability is controlled at no more than α.
Type II error: failing to reject H₀ when H₀ is false. Its probability is β for a specified alternative; power is 1 − β.

The two errors live on opposite sides of the same decision. For a fixed design and alternative, lowering α usually reduces power and raises β. Sample size, effect size, measurement quality, and the decision rule also matter; α alone does not determine β.

The Asymmetry Is Always There

Different domains weight the two error types differently:

Cancer screening: a Type I (false-positive screening) is the cost of unnecessary follow-up tests and anxiety; a Type II (missed cancer) is far worse. Screening tests are calibrated toward sensitivity (low Type II) at the cost of more Type I.
Drug approval: a Type I (approving a drug that doesn't work) wastes money and creates side effects in patients; a Type II (rejecting an effective drug) means patients suffer without treatment. Regulators use multi-stage evidence and explicit benefit–risk judgments; the decision is not captured by α alone.
Spam filtering: a Type I (legitimate email marked as spam) means a missed message; a Type II (spam in the inbox) means clutter. Most filters are tuned toward Type II (more clutter, fewer missed messages) because missing a real email is more costly than tolerating spam.
Criminal trial: a Type I (convicting an innocent person) is treated as worse than a Type II (acquitting a guilty person). 'Beyond reasonable doubt' reflects a special concern about wrongful conviction, but it is not a calibrated numerical α.

The Blackstone Ratio

The 18th-century English jurist William Blackstone wrote: 'it is better that ten guilty persons escape than that one innocent suffer.' The maxim expresses a normative priority against wrongful conviction, not an empirical 10:1 exchange rate or a complete model of the common-law tradition.

Where Track 06 Picks This Up

Track 06 uses this asymmetry as a lens for the courtroom, while keeping the analogy's limits visible. A complaint about “letting a monster walk” may ignore the cost of wrongful conviction, but an acquittal does not establish that a guilty person escaped, and legal judgment cannot be reduced to one error-rate calculation.

Code

Type I and Type II error rates, simulated·python

import numpy as np
rng = np.random.default_rng(150)

# Simulate a test for a true difference.
# H0: no effect (mean = 0).  H1: true effect (mean = 0.3).
# Sample size N, significance level alpha.
N = 30
alpha = 0.05
threshold = 1.96 / np.sqrt(N)   # critical value for sample mean

M = 10_000
true_means = [0.0, 0.3]   # null world and effect world
for true_mean in true_means:
    samples = rng.normal(loc=true_mean, scale=1.0, size=(M, N))
    sample_means = samples.mean(axis=1)
    rejected = np.abs(sample_means) > threshold
    fraction = rejected.mean()
    label = "true H0" if true_mean == 0 else "true H1 (effect=0.3)"
    print(f"{label:>30s}: rejection rate = {fraction:.4f}")

# Under true H0: rejection rate ≈ alpha = 0.05 (Type I error rate).
# Under true H1 (effect 0.3, N=30): rejection rate = power = 1 - Type II rate.
# To lower Type I, raise the threshold — but that lowers power (raises Type II).
# The trade-off is fundamental for a fixed N.

Exercise

Pick a decision in your life with two failure modes (acting on a false alarm vs missing a real signal). Examples: replying to a possibly-phishing email, taking medication based on a possibly-incorrect diagnosis, leaving early for an appointment because traffic might be bad. Which Type — I or II — are you more averse to in each case? The answer reveals your implicit α; once you see it, you can ask whether it's calibrated to the actual costs.

Hint

Most of your daily decisions have a Type I and a Type II side. The skill is naming them, weighing their costs, and choosing an α that matches the weighting — rather than defaulting to 'whatever feels right in the moment.'

The Definitions

The Asymmetry Is Always There

The Blackstone Ratio

Where Track 06 Picks This Up

Code

External links

Exercise

Progress

Comments 0