Cost and Latency Napkin Math

The numbers you actually need

You don't need a finance degree to estimate LLM costs — you need three numbers per model and three operations.

The three numbers per model

Cost per million input tokens (provider-published).
Cost per million output tokens (provider-published).
Tokens per second (provider-published or measured).

For reasoning models, output tokens include thinking tokens — make sure you understand whether thinking tokens are billed and at what rate.

The three operations

Per-request cost. input_tokens × input_rate + output_tokens × output_rate. For reasoning models, add (thinking_tokens × thinking_rate).

Per-request latency. output_tokens / TPS. For reasoning models, this includes thinking time, which can dominate.

Throughput at scale. Concurrent requests × TPS, bottlenecked by provider capacity.

Worked example — agent making 50 calls per task

Take an agent workflow that makes 50 LLM calls per user task, with average 1K input + 500 output tokens per call. With a fast non-reasoning model at $1/M input, $3/M output, that's roughly 50 × ($0.001 + $0.0015) = $0.125 per task. Switch the same agent to a reasoning model that adds 4K thinking tokens per call at $15/M output: 50 × ($0.001 + $0.0015 + $0.060) = $3.13 per task — a 25× cost increase. This is why agent workflows almost always avoid reasoning mode.

Worked example — a 'careful' analysis task

One question, allowed to think for 16K tokens: $15/M output × 0.016 = $0.24 just in thinking. Add 2K of final answer + 4K input: total maybe $0.28. For a hard analysis task this is often acceptable, especially if the answer would have taken 10+ minutes of human work.

The general framing

The cost/latency model splits cleanly into three knobs: input length (cheap), output length (more expensive), thinking length (most expensive on most APIs). Optimization usually means cutting the longest leg — and on reasoning workloads, that's almost always thinking length first.

Code

Cost estimator for a reasoning model call·python

def estimate_cost(input_tokens, output_tokens, thinking_tokens=0,
                  rate_in=1.0, rate_out=3.0, rate_think=15.0):
    # Rates per 1M tokens, all in dollars.
    return (input_tokens * rate_in
            + output_tokens * rate_out
            + thinking_tokens * rate_think) / 1e6

Latency estimator (sequential, single request)·python

def estimate_latency_sec(output_tokens, thinking_tokens, tps=80):
    # Assumes thinking + output tokens are generated sequentially at TPS.
    return (thinking_tokens + output_tokens) / tps

Exercise

Pick a real LLM-using workload from your project. Estimate per-request cost using the three knobs (input, output, thinking). Now estimate the cost difference if you switched between fast mode and reasoning mode for the same workload. The number you get is the budget impact of one design decision — and it is usually larger than people expect.