Step Compression — Don't Show Your Work When You Don't Have To

~12 min · reasoning, compression, cost

Level 0Apprentice

0 XP0/100 lessons0/14 achievements

0/120 XP to next level120 XP to go0% complete

Reasoning tokens are tokens

Once you've validated that a prompt works with reasoning, you can often compress the reasoning step out — either by switching to a non-reasoning model, lowering reasoning effort, or stripping CoT from the prompt. The reasoning was scaffolding; once the prompt is right, the scaffolding may not be load-bearing.

The compression test

Run with full reasoning. Capture outputs and accuracy.
Run with reasoning off (or 'low'). Compare.
If accuracy is unchanged, compress.
If accuracy drops on a specific subset, gate reasoning on that subset only (cheap classifier in front of the call).

Why this matters

Reasoning costs scale with traffic. Production systems running reasoning on every call when most calls don't need it are paying tens of thousands of dollars per month for marginal accuracy. Routing is the lever.

Code

Routing reasoning by request type·python

def needs_reasoning(req) -> bool:
    # cheap classifier or rule-based router
    return req.complexity_score > 0.7 or req.task in {"plan", "debug"}

resp = client.messages.create(
    model="claude-opus-4-7",
    thinking={"type": "enabled", "budget_tokens": 8_000} if needs_reasoning(req) else None,
    messages=req.messages,
)

External links

Anthropic — Cost optimization

Exercise

Identify 100 production requests where you currently use reasoning. Classify them as 'needed' or 'overkill.' Build a routing rule that uses reasoning only on 'needed' requests. Measure cost change.