C.W.K.
Stream
Lesson 02 of 05 · published

Budget, Not Trash Bin

~24 min · budget, safety-margin, output

Level 0Window Watcher
0 XP0/50 lessons0/13 achievements
0/100 XP to next level100 XP to go0% complete

The wrong instinct

The common beginner move is to keep piling material into the context until the model complains. That treats the window like a storage bin. Real work treats it like a budget.

Every token has an opportunity cost. A stale tool dump competes with fresh requirements. A pasted design doc competes with a failing test log. A long apology from the model competes with the next command output. The question is never "can it fit?" The question is "does it deserve attention right now?"

Safety margin is part of the design

Do not drive the window to the wall. Keep a margin for output, corrections, tool results, and the unexpected turn where the human says, "actually, this rule matters more." When the margin disappears, the model becomes brittle: it stops reasoning carefully, drops citations, truncates final answers, and starts agreeing with everything because there is no room left to push back.

Pre-allocate before loading

The mature pattern is to reserve output, reasoning, and a safety buffer first, then decide how much input fits. Most loops do this in reverse, then act surprised when the answer ends mid-sentence.

Iron rule: never wait until the context is full to checkpoint. Full context means no maneuvering room — and zero room is exactly when the hardest turn always shows up.

Code

Safety-margin policy·yaml
context_budget:
  hard_limit: 100%
  checkpoint_at: 65%
  compact_or_handoff_at: 75%
  emergency_only: 85%
  never_plan_for: 100%
Reserve before loading·python
WINDOW = 200_000
RESERVE_OUTPUT = 16_000
RESERVE_REASONING = 12_000
SAFETY_MARGIN = 16_000

usable_input_budget = WINDOW - RESERVE_OUTPUT - RESERVE_REASONING - SAFETY_MARGIN
assert usable_input_budget > 0
print(f"You may load up to {usable_input_budget} input tokens.")

External links

Exercise

Define a context budget policy for your own AI workflow. Choose checkpoint, compaction, and stop thresholds.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.