Skip to content
C.W.K.
Stream
Lesson 04 of 04 · published

What the Measurements Changed

~13 min · measurements, plan, compaction, fidelity

Level 0Cold Stick
0 XP0/41 lessons0/14 achievements
0/100 XP to next level100 XP to go0% complete

Numbers That Earned a Gate

Late August 2026, same two-step filesystem task, fixed seeds, a local Qwen on a family daemon. Loop finished 3/3 in four steps and about seven thousand tokens. Plan invented six steps, hit the step cap, cost about six times as much, and left a run incomplete. After the plan learned an executable budget from that turn's own caps — a plan step costs about two model steps, and the instruction forbids splitting work one tool call finishes — planned steps fell to about 1.7, the cap vanished, success went 3/3. Plan was then correct and still about 2.8× the loop. Loop stayed the local default. Plan earns its cost when the numbered list is the deliverable.

Compaction was hijacked by conversational momentum: the summarizer, given the pending operator prompt, answered the task instead of summarizing. The fix excludes that trailing prompt, compares estimate against estimate so a fresh marker does not look stale, and falls through to an ordinary step after one failed attempt. A later staged crossing summarized once and recalled four planted facts, three of them only in the summary. The gate has a remaining blind spot for a single result that jumps past the window; that is named, not wished away.

A cell carries its daemon version. Copying last month's 3/3 forward is how a rotting blob keeps a trophy.

Fidelity Is Per Release

The same small-model battery on daemon 0.33.0: a 27B Qwen 3/3 with no tool errors; the latest Qwen tag 3/3; a 31B Gemma 3/3 after the previous daemon had eaten retries; a 20B mixture-of-experts still 0/3 because the daemon 500s on a year-old blob that ships no tool capability. The upgrade cost nothing and bought competence back on the weaker arms. The model that still fails has not earned the bench by being on disk.

The mini door cost about twice the tokens on short tool tasks and changed competence not at all — 3/3 both arms. Attach it for the voice. Same-model delegate was 2/2 on a healthy pair and failed typed when the child named a broken model — no mutation, honest report. Those rows changed instructions, gates, and defaults. They did not become a leaderboard.

Code

Rows that changed the product, not a ranking·text
loop vs plan     4.0 steps / ~7k tok  vs  still ~2.8x after budget gate
compaction       pending prompt excluded; estimate vs estimate
mini door        ~2x tokens, competence 3/3 == bare 3/3
small models     number + daemon version; on-disk ≠ earned
delegate         2/2 healthy; typed fail on a broken child model

External links

Exercise

Rewrite one of the rows above as a trophy sentence, then as a notebook sentence. Keep the notebook sentence. Throw the trophy away.
Hint
The trophy uses 'better.' The notebook uses a denominator, a daemon version, and a stop reason.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.