Reasoning Budget vs Output Budget

~12 min · reasoning, budget, max-tokens

Level 0Apprentice

0 XP0/100 lessons0/14 achievements

0/120 XP to next level120 XP to go0% complete

Two budgets, two failure modes

Reasoning models bill thinking tokens separately from output tokens. The thinking budget is invisible to the user but very visible on the bill. Conversely, if your output max_tokens is too small, the model truncates the answer mid-sentence. The two budgets need to be set deliberately.

Setting the thinking budget

Trivial classification — off or 'low.' A few hundred tokens at most.
Multi-step planning — 4k–10k tokens. Gives room for branches.
Hard math, deep code analysis — 20k+ tokens. Test that more budget actually helps; diminishing returns set in.

Setting the output budget

Match max_tokens to the expected output shape with margin. JSON object with five fields → 512 max_tokens is plenty. Long-form essay → 2k–4k. Truncated outputs are a recurring incident pattern; make max_tokens explicit and add an alert when responses hit the cap.

Code

Two-budget call·python

client.messages.create(
    model="claude-opus-4-7",
    thinking={"type": "enabled", "budget_tokens": 8_000},
    max_tokens=1_024,  # output budget
    messages=[...],
)

External links

Exercise

Audit a production prompt's two budgets. Lower thinking by half on a sample of requests; measure quality. Add an alert when output max_tokens is hit.

Progress

Progress is local-only — sign in to sync across devices.

← PreviousStep Compression — Don't Show Your Work When You Don't Have To Next →The Trap of Reasoning Theatre

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.