C.W.K.
Stream
Lesson 05 of 05 · published

Quality Is Advisory

~10 min · quality, advisory, not-proof, gate

Level 0Empty Shelf
0 XP0/35 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete
"A clean-looking transcript can be perfectly, fluently wrong."

What the Quality Check Actually Measures

When Recall accepts a transcript, it runs a structural quality assessment. It's genuinely useful — it catches the tell-tale shapes of a broken transcription:

  • Repeated segments — the same line 40 times, the classic signature of a model looping on silence.
  • Long repeated spans — a phrase that runs on far past where any human would stop.
  • Timestamp regression — timestamps that go backwards, meaning the projection is confused.
  • Known stock phrases — the boilerplate hallucinations a model emits when it hears nothing ("thanks for watching," subtitle-credit filler).

These are real signals. A release flagged severe can even be blocked from generating a summary, because summarizing garbage produces confident garbage. So the quality check earns its place.

But It Measures Shape, Not Truth

Here's the line you must not cross: structural quality measures the shape of the text, never its correctness. The two come apart in both directions:

  • A transcript can be structurally spotless — no repeats, clean timestamps, no stock phrases — and still be wrong. The model can mishear a name perfectly fluently. Nothing in the shape reveals that.
  • A transcript can be structurally messy — a few repeats, one odd span — and still be completely usable for finding the moment you're searching for.

So the quality score is advisory. It informs a human reviewer and can gate an expensive derived step. It is never a proof of correctness, and — this is the invariant the next track is built on — it never auto-promotes a transcript to 'good enough to be knowledge.' A clean structural score and 'trustworthy as a source' are two different judgments, and the machine only gets to make the first one.

Why This Restraint Matters

It would be so tempting to let a green quality score mean 'approved.' It's automatic, it's cheap, it feels objective. That's exactly the trap. The moment a proxy metric becomes the approval gate, you've quietly redefined 'correct' as 'passes the structural check' — and now every fluent hallucination sails straight through. Recall keeps the metric in its lane: a helpful advisor that a human can overrule in either direction, never a judge. Hold that distinction; Track 7 turns it into the hard wall between 'complete' and 'approved.'

Code

Quality gates an expensive step; it never approves truth·python
assessment = {
    'flags': ['repeated_segment', 'stock_phrase'],
    'severity': 'severe',  # advisory only
}

# LEGITIMATE use: gate an expensive derived step.
if assessment['severity'] == 'severe':
    skip_summary_generation(release)   # don't summarize garbage

# FORBIDDEN use: treat a clean score as approval.
if assessment['severity'] == 'clean':
    approve_for_corpus(release)   # NO. shape != truth.
                                  # a fluent mishearing is 'clean'.
# Structural quality informs a human; it never auto-approves.

External links

Exercise

Find an automated quality score in your own world — a lint pass, a coverage percentage, a health check, a spam score. Ask two questions: does a passing score prove the thing is actually correct, and is that score used anywhere as an automatic approval? If a green score silently approves something, find the case where the shape is clean but the substance is wrong, and add a human judgment step for the decision that actually matters.
Hint
The dangerous pattern is 'score is green, therefore ship/approve/trust.' Separate the two roles: let the metric block obvious garbage and inform a reviewer, but route the real approval — 'is this actually good enough for its purpose?' — through a judgment the metric only advises. Clean shape and trustworthy substance are different questions.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.