C.W.K.
Stream
Lesson 06 of 06 · published

Closer: Tools Are Honest About Their Preconditions

~8 min · closer, synthesis, preconditions, cross-ref-track-06

Level 0Stats Novice
0 XP0/55 lessons0/14 achievements
0/100 XP to next level100 XP to go0% complete
"The frequentist toolkit is powerful and honest about its preconditions. Citizens who use it without checking the preconditions are using a powerful tool dishonestly."

What This Track Established

Six lessons, one toolkit. Confidence intervals (long-run hit rates of an interval-construction procedure), hypothesis tests (a structured procedure for asking 'is the evidence strong enough to overturn the default?'), p-values (P(data | null), not P(null | data)), p-hacking (selection bias inside the analysis pipeline), and the Type I / Type II asymmetry (the policy decision baked into every test). Together these are the frequentist workshop.

Every tool rests on the same preconditions the CLT named: independence and finite variance, plus large enough N for the bell to apply. When these hold, the tools are well-defined and operationally useful. When they don't, the tools still output numbers — but the numbers are arithmetic without scientific meaning.

The Bridge to Track 06

Track 06 turns the hypothesis-test frame into its citizen-relevant application: the courtroom. There:

  • H₀ = innocent.
  • H₁ = guilty.
  • 'Beyond reasonable doubt' = α set very, very small.
  • The Type I / Type II asymmetry is at maximum strength — convicting the innocent is catastrophic; acquitting the guilty is acceptable cost.

The 'why are they letting that monster walk?' frame fixates on Type II error (the guilty going free) and is blind to the deliberate, principled Type I aversion the system was built around. Once the asymmetry is named, the complaint dissolves. The system did what it was designed to do.

The Bridge to Track 08

Track 08 (Bayesian Frame) takes the same hypothesis-testing puzzle and runs it through a Bayesian engine. P(guilty | evidence) becomes computable, but only with a prior P(guilty) — which the legal system carefully avoids stating explicitly because it would be politically toxic. The frequentist frame avoids the prior problem by refusing to compute P(guilty | evidence) at all; the Bayesian frame embraces it and forces honesty about priors. Both frames are legal; the dialog between them is the citizen's skill.

Carry Forward

The frequentist toolkit is honest about what it does and does not say. Most citizen misuse comes from reading its outputs as answers to questions it does not answer. P-value is not P(null is true). CI is not 'probability the parameter is in here.' Statistical significance is not the size or importance of an effect. Hold these distinctions and you've already separated yourself from the modal misreading. Track 06 turns the toolkit on the courtroom; Track 08 inverts the frame entirely. The dialog between the three is the lens the rest of the quest sharpens.

Code

Frequentist output cheat-sheet: IS vs IS NOT·python
# Synthesis cheat-sheet: what each frequentist output is, and is NOT.
print("""
FREQUENTIST OUTPUT      IS                                NOT
---------------------   --------------------------------- ----------------------------------
p-value                 P(data this extreme | H0 true)    P(H0 true | data)
95% CI                  long-run 95% hit rate of method   '95% probability the param lies here'
'significant at 0.05'   tolerance for false alarms = 5%   evidence the alternative is true
Type I error rate (a)   P(reject H0 | H0 true)            P(H0 true given rejection)
Power (1 - b)           P(reject H0 | H1 true)            'how likely the effect is real'
""")

# Print this cheat-sheet and re-read it any time you find yourself making
# a statistical claim from a number. The 'IS NOT' column is the one citizens
# silently invoke and is the one the next two tracks (06 and 08) dismantle.
""""""

External links

Exercise

Make a list of every statistical claim you encounter over the next 24 hours (news, podcasts, papers, ads, conversations). For each, ask: is this a p-value, a CI, an effect size, or a vague 'significant' claim? For each, restate it precisely using the cheat-sheet's IS column. Notice how many statements quietly slide into the IS NOT column when said in everyday language.
Hint
Most everyday statistical talk is in the IS NOT column. The discipline is to recognize the slip without becoming a pedant — the goal is calibration, not nitpicking.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue
💛 by Ttoriwarm

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.