Sort Lessons by Lifetime, Not by Severity

The Problem With Keeping Everything

A log that records every verdict grows without bound, and a document nobody can finish reading transmits nothing. The obvious fix is to keep only the important entries — but importance is the wrong axis, because a trivial-sounding verdict can contain a lesson that applies forever, while a dramatic one can be entirely local to a single run.

The axis that works is durability: how long is this true for? Sorting on that gives three destinations, and each one has a different lifetime.

Three Destinations

Per-run verdicts stay with the run. "This image was too dark, regenerated" is true about one asset in one episode. It lives in that run's log, in the working directory, and it is discarded when the directory is. That is correct — it was never knowledge, it was bookkeeping, and keeping it forever would bury the entries that matter.

Lessons that generalize become code. If a verdict would apply to any future run, it does not belong in prose at all. It becomes a check that fails the build. The subtitle-collision geometry, the invented-day-boundary detector, the per-language audio measurement — each began as somebody noticing something in one run, and each is now a check nobody has to remember.

Summary verdicts go on the request thread. The handful of judgments that explain what this piece is and why it took the shape it did belong with the conversation that commissioned it, where a person will actually encounter them.

The Consequence: the Tool Gets Smarter, the Log Does Not Get Longer

This is the property the split is designed to produce, and it is worth stating as a goal rather than as a side effect. Under the naive approach, everything learned accumulates in a document, which grows until reading it is a project, at which point it stops being read and the learning stops transferring. Under the durability split, the general lessons leave the document entirely — they become behavior — and the document stays a manageable record of one run.

A pipeline that has run twenty times has twenty short logs and a tool full of checks. The alternative is one enormous document and a tool that has learned nothing.

A lesson that stays prose will be re-learned. Anything you write down as guidance depends on someone reading it at the right moment, which is a coin flip at best. If a lesson can be expressed as a check that fails, express it that way — the document is where lessons go to be forgotten politely.

Code

Routing a verdict by how long it stays true·text

verdict: "the script says 'yesterday' about turns three minutes apart"

  is this true about ONE run?              no - any run can do this
  can it be expressed as a check?          yes - measure the real span
  ---------------------------------------------------------------
  -> PROMOTE. becomes a hard check in the pipeline.


verdict: "candidate 3 had a distracting background, used 2 instead"

  is this true about ONE run?              yes - one asset, one episode
  can it be expressed as a check?          no - it is a taste judgment
  ---------------------------------------------------------------
  -> STAYS WITH THE RUN. discarded with the working directory.


verdict: "we cut the politically adjacent opening; the argument survives"

  is this true about ONE run?              yes
  does it explain the shape of the piece?  yes
  ---------------------------------------------------------------
  -> SUMMARY. goes on the request thread with the delivery.

What promotion actually looks like·python

# The pilot narrated "yesterday" about exchanges minutes apart,
# because the source material used that word and it was inherited
# without checking. Prose guidance would have been forgotten.
# Instead:

DAY_WORDS = ("yesterday", "어제", "last night", "the next day")

def check_invented_day_boundaries(script: dict, span_hours: float) -> list[str]:
    """Turn-to-turn time is not calendar time.

    A conversation is one session unless a date says otherwise.
    Day-boundary language against a sub-20h span is a fabrication.
    """
    if span_hours >= 20 or script.get("spans_days"):
        return []                       # genuinely crosses days; allowed
    return [
        f"{seg['key']}: day-boundary language over a {span_hours:.1f}h"
        f" conversation - {word!r} in {seg['text']!r}"
        for seg in script["segments"]
        for word in DAY_WORDS
        if word in seg["text"].lower()
    ]

# One run's mistake is now every future run's guardrail,
# and nobody has to have read the log to benefit from it.

Exercise

Open your team's most recent few incident reports or retrospectives and list the action items. Sort each by durability: local to that incident, or true for every future run? For the general ones, check whether they became code, a test, or an alert — or whether they became a sentence in a document. Count the ratio; it predicts how many of those incidents will recur.

Hint

Action items phrased as "be careful to" or "remember that" are prose lessons wearing an action item's clothes. Each one is a promotion that was attempted and abandoned, and rewriting even one of them as a check is usually worth more than the whole rest of the list.

The Problem With Keeping Everything

Three Destinations

The Consequence: the Tool Gets Smarter, the Log Does Not Get Longer

Code

External links

Exercise

Progress

Comments 0