Skip to content
C.W.K.
Stream
Lesson 06 of 06 · published

Closer: Tools Are Honest About Their Preconditions

~8 min · closer, synthesis, preconditions, cross-ref-track-06

Level 0Stats Novice
0 XP0/55 lessons0/14 achievements
0/100 XP to next level100 XP to go0% complete
"The frequentist toolkit is powerful and honest about its preconditions. Citizens who use it without checking the preconditions are using a powerful tool dishonestly."

What This Track Established

Six lessons, one toolkit. Confidence intervals (long-run hit rates of an interval-construction procedure), hypothesis tests (a structured procedure for asking 'is the evidence strong enough to overturn the default?'), p-values (P(data | null), not P(null | data)), p-hacking (selection bias inside the analysis pipeline), and the Type I / Type II asymmetry (the policy decision baked into every test). Together these are the frequentist workshop.

These tools do not all rest on one bell-curve assumption. Their validity depends on the sampling design, model, estimator, test statistic, and error-control procedure. Normal approximations are common, but exact, randomization, robust, and resampling methods have different conditions.

The Bridge to Track 06

Track 06 uses a bounded courtroom analogy: the presumption of innocence resembles a protected null position, and wrongful conviction resembles a false positive. The legal burden has no fixed α, and an acquittal does not identify a factual false negative.

The 'why are they letting that monster walk?' frame may ignore the special weight given to avoiding wrongful conviction. But the comparison remains an analogy: acquittal is not proof of factual guilt, and criminal procedure is not a repeated statistical test.

The Bridge to Track 08

Track 08 uses Bayes' rule to show why a likelihood cannot be inverted into a posterior without a prior and a complete evidence model. That mathematical lesson does not imply that a court should assign defendants demographic “priors of guilt.” Legal proof and statistical inference answer related but non-identical questions.

Carry Forward

The frequentist toolkit is honest about what it does and does not say. Most citizen misuse comes from reading its outputs as answers to questions it does not answer. P-value is not P(null is true). CI is not 'probability the parameter is in here.' Statistical significance is not the size or importance of an effect. Hold these distinctions and you've already separated yourself from the modal misreading. Track 06 compares the toolkit with legal decision-making; Track 08 adds prior-and-posterior questions. The dialog between the three is the lens the rest of the quest sharpens.

Code

Frequentist output cheat-sheet: IS vs IS NOT·python
# Synthesis cheat-sheet: what each frequentist output is, and is NOT.
print("""
FREQUENTIST OUTPUT      IS                                NOT
---------------------   --------------------------------- ----------------------------------
p-value                 P(data this extreme | H0 true)    P(H0 true | data)
95% CI                  long-run 95% hit rate of method   '95% probability the param lies here'
'significant at 0.05'   tolerance for false alarms = 5%   evidence the alternative is true
Type I error rate (a)   P(reject H0 | H0 true)            P(H0 true given rejection)
Power (1 - b)           P(reject H0 | H1 true)            'how likely the effect is real'
""")

# Print this cheat-sheet and re-read it any time you find yourself making
# a statistical claim from a number. The 'IS NOT' column is the one citizens
# silently invoke and is the one the next two tracks (06 and 08) dismantle.
""""""

External links

Exercise

Make a list of every statistical claim you encounter over the next 24 hours (news, podcasts, papers, ads, conversations). For each, ask: is this a p-value, a CI, an effect size, or a vague 'significant' claim? For each, restate it precisely using the cheat-sheet's IS column. Notice how many statements quietly slide into the IS NOT column when said in everyday language.
Hint
Most everyday statistical talk is in the IS NOT column. The discipline is to recognize the slip without becoming a pedant — the goal is calibration, not nitpicking.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue
💛 by Ttoriwarm

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.