Stable Prefix, Variable Tail

~25 min · prompt-caching, prefix, structure

Level 0Window Watcher

0 XP0/50 lessons0/13 achievements

0/100 XP to next level100 XP to go0% complete

Caches reward sameness

Prompt caching works best when stable content appears at the beginning and variable content appears later. Instructions, tool schemas, reusable examples, and stable corpus material belong in the prefix. Current user questions, fresh diffs, timestamps, and run-specific IDs belong in the tail.

Two effects, one rule

This is not only a cost trick. It also reinforces good context hygiene: stable rules stay stable, live inputs stay obviously live. The cache-friendly shape and the attention-friendly shape happen to be the same shape — that is not coincidence, it is good API design.

Structure for the next hundred turns

A cache-friendly prompt is easier to reason about, easier to diff, and easier to debug when behavior changes. Designing for the next single turn is a beginner mistake; designing for the next hundred turns is the sustained-cost win.

Code

Cache-aware prompt shape·yaml

prompt:
  stable_prefix:
    - system rules
    - tool definitions
    - style contract
    - reusable examples
    - corpus snippets re-read often
  variable_tail:
    - current user request
    - latest diff
    - latest tool result
    - timestamp if truly needed

Anthropic cache_control example·python

messages = [{
    "role": "user",
    "content": [
        {"type": "text", "text": SYSTEM_RULES,
         "cache_control": {"type": "ephemeral"}},
        {"type": "text", "text": TOOL_SCHEMAS,
         "cache_control": {"type": "ephemeral"}},
        {"type": "text", "text": current_question},
    ],
}]

External links

Exercise

Take one long prompt and split it into stable prefix and variable tail. Move anything volatile out of the prefix. Add a cache_control breakpoint at the prefix boundary if your provider supports one.

Hint

If it changes every request, it does not belong in the prefix.

Progress

Progress is local-only — sign in to sync across devices.

← 🗂️ Library Strategy & RAGQuiz · 5 questions Next →Cache Busters: The Top Five

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.