Cache Math: Worked Examples

Numbers make the case

The argument for caching becomes obvious as soon as you do the math on a real workflow. Take a 50K-token codebase review producing 8K tokens of feedback at illustrative Sonnet pricing ($3/M input, $15/M output, $0.30/M cached input).

Three scenarios

Scenario A — single shot: 50K input × $3/M + 8K output × $15/M = $0.27.

Scenario B — 10 follow-up turns, no caching: history grows each turn (~50K → ~100K), input dominates → ~$3.50 input + ~$1.20 output = ~$4.70.

Scenario C — 10 follow-up turns, with caching: stable prefix (~45K) cached, fresh input ~5K per turn → ~$0.40 input + ~$1.20 output = ~$1.60. Same conversation, 3x cheaper.

Where caching pays back fastest

Caching pays back fastest in workflows with: a large stable prefix (rules + tool schemas + corpus), many short follow-up turns, and short variable tail. Coding agents, customer-support bots, and long research chats are the canonical wins. Single-shot one-off prompts barely benefit.

Code

Cache math estimator·python

def estimate(prefix_tokens, tail_tokens, output_tokens, turns,
             p_input=3.0, p_cached=0.30, p_output=15.0):
    # All prices per million tokens.
    no_cache = (prefix_tokens + tail_tokens) * turns / 1e6 * p_input \
             + output_tokens * turns / 1e6 * p_output
    with_cache = (prefix_tokens / 1e6 * p_input                     # first turn full price
                 + prefix_tokens * (turns - 1) / 1e6 * p_cached     # next turns cached
                 + tail_tokens * turns / 1e6 * p_input              # tail always fresh
                 + output_tokens * turns / 1e6 * p_output)
    return {"no_cache": round(no_cache, 2), "with_cache": round(with_cache, 2)}

Sample run·python

estimate(prefix_tokens=45_000, tail_tokens=5_000,
         output_tokens=2_000, turns=10)
# -> { "no_cache": 1.80, "with_cache": 0.61 }  (illustrative)

Exercise

Pick one of your real long-running AI workflows. Plug realistic prefix/tail/output token counts and turn counts into the estimator. Compute the cost with and without caching. Decide whether to invest in cache structure.

Hint

If the cached cost is 30%+ less than the no-cache cost, caching is essentially free money.