Prompt Caching Strategy in Depth

Four breakpoints, used wisely

You get up to four cache_control: ephemeral breakpoints per request. The classic split: (1) system prompt with persona / policy, (2) tools list, (3) static documents (RAG context that does not change for this session), (4) early conversation history. Each breakpoint cuts where the next thing changes most often.

TTL and the 5-minute window

The default ephemeral cache TTL is 5 minutes. For workloads where calls are bursty, this is enough. For workloads where calls are sparse, you pay the cache write more often than you read it — measure before claiming caching helps.

Order matters more than people think

The cache rewards a stable prefix. If the assistant's last response is the same length but slightly reordered text, the cache hit is lost. Treat your prompt construction as deterministic — same inputs in, same byte-for-byte prefix out.

Principle: Cache stable prefixes. Make the front of your prompt boring on purpose. Verify cache hits with usage telemetry, not faith.

Code

Four-breakpoint layout·python

resp = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    system=[
        {"type": "text", "text": PERSONA, "cache_control": {"type": "ephemeral"}},  # bp 1
    ],
    tools=[
        # ... last tool gets bp 2
        {**LAST_TOOL_SPEC, "cache_control": {"type": "ephemeral"}},
    ],
    messages=[
        {"role": "user", "content": [
            {"type": "text", "text": SESSION_DOCUMENT, "cache_control": {"type": "ephemeral"}},  # bp 3
        ]},
        # ... older history
        {"role": "assistant", "content": [
            {"type": "text", "text": LAST_STABLE_ASSISTANT, "cache_control": {"type": "ephemeral"}},  # bp 4
        ]},
        # newest user turn (uncached, dynamic)
        {"role": "user", "content": current_user_text},
    ],
)

Verify cache hit ratio in production·python

def cache_hit_ratio(usage) -> float:
    total_input = usage.input_tokens + usage.cache_creation_input_tokens + usage.cache_read_input_tokens
    if total_input == 0:
        return 0.0
    return usage.cache_read_input_tokens / total_input

# Aggregate across calls; alert if the ratio drops
# (e.g., a deployment changed the persona file and broke the cache prefix).

Prompt Caching Strategy in Depth

Four breakpoints, used wisely

TTL and the 5-minute window

Order matters more than people think

Code

External links

Exercise

Progress

Comments 0