Output and Reasoning Tokens Still Count

Input is only half the bill

The context window is not just what you send. It also has to contain what the model writes back. Some models also spend reasoning tokens that count against the request budget even when they are not displayed directly. If you ask for a 30-page rewrite after stuffing the entire window with source material, you have designed a failure.

Reasoning models change the math

OpenAI o-series and Claude extended-thinking models can emit thousands of hidden reasoning tokens before answering. Some providers bill them; some do not. All of them count against the per-request token cap. A request that 'should' fit can fail mid-generation because reasoning consumed the headroom.

Reserve output explicitly

For long tasks, decide the output budget first. Then decide how much source material can fit. This feels backwards until you watch a model truncate the final answer right before the important part — and then it feels obvious.

Truncation rule: a long input with no output reserve is not careful. It is a delayed truncation bug pretending to be a successful request.

Code

Reserve before loading·python

WINDOW = 400_000
RESERVE_OUTPUT     = 40_000
RESERVE_REASONING  = 30_000   # extended-thinking budget
SAFETY_MARGIN      = 40_000

USABLE_INPUT = WINDOW - RESERVE_OUTPUT - RESERVE_REASONING - SAFETY_MARGIN
print(USABLE_INPUT)  # 290_000

Two budget templates·yaml

budgets:
  small_window_128k:
    input:     80_000
    output:    20_000
    reasoning: 12_000
    margin:    16_000
  long_window_400k:
    input:     260_000
    output:    50_000
    reasoning: 40_000
    margin:    50_000

Output and Reasoning Tokens Still Count

Input is only half the bill

Reasoning models change the math

Reserve output explicitly

Code

External links

Exercise

Progress

Comments 0