A System Prompt That Holds Still

One Byte Is Enough

Every provider the engine talks to can cache a prompt it has already seen, and charges less and answers faster for the part that matches. What counts as matching differs. Z.ai, which serves the GLM brain, matches the whole system field exactly; a probe showed that one byte changing per turn forfeits about 100K tokens of cache. Anthropic, OpenAI and Google match prefixes, where a changed byte costs everything after it. Either way the rule is strict.

So since 2026-08-14 (coop #117 and #119) the engine keeps a contract: the system prompt is byte-identical across every turn of a conversation. Nothing volatile may go there, and the prefix-stability tests fail loudly if someone adds it. The contract assumes the conversation's own settings hold still; the end of this lesson shows where Master Chief does not.

Where the Moving Parts Go

Plenty does change per turn. All of it rides the current user message instead, prepended to Dad's words in this order:

Per-turn content	Why it moves
The clock	minute resolution
The conversation timeline	the gaps between turns grow
Retrieved memory	shaped by this turn's words
Briefs the memory librarian finished in the background	they arrive whenever they are done (lesson 3)
Where Dad was when he sent it	the phone attached a location fix
Voice blocks	this turn was dictated, or wants a spoken reply

That slot is sent and never persisted. The stored row keeps Dad's words alone, so replaying an old turn never drags along an old clock or a stale retrieval. Voice and text switch freely inside one conversation for the same reason: the system prompt never learns which kind of turn this is.

Moving retrieval into a user position had a cost of its own. Text inside a user message reads as something the speaker just said, so the retrieved block is headed in so many words: reference material, background, not the current task. That framing helped at the end of system; in the user slot it is load-bearing.

What It Bought, Measured

The commit that moved the clock and retrieval out measured a real route on the GLM brain, with 65-second gaps so the minute ticked between turns, which used to be fatal. Turn one paid 109,071 tokens cold. Turns two and three each read 108,224 from cache, leaving about 850 uncached tokens per turn: the clock, the retrieval, and the newest history. The day before, every human-paced turn paid the whole ~100K cold.

Two Five-Minute Clocks

Don't confuse the engine's cache with the provider's. The engine keeps its vault-built base for five minutes so it does not reread hundreds of files every turn. When that expires it rebuilds, and an unchanged vault rebuilds the same bytes. An edited vault reaches the prompt within five minutes, and that one turn pays a cold prefix: a price paid on purpose, once per edit. The provider's cache lifetime is the provider's business. Anthropic's default also happens to be five minutes; coincidence, not design.

The Exception, Told Plainly

Master Chief bends this contract, and you should know where. Its route appends a system addendum to every Chief turn: the Chief's standing instructions plus one line, Current origin:, naming the doorway this turn came through (the surface and the app thread). The chat route joins that addendum onto the system prompt. So in Master Chief the system field changes whenever the doorway changes between turns: phone, then browser, then Telegram.

Neither the Chief's design doc nor the engine's docstring mentions it; reading the code for this quest is what surfaced it. Treat it as a known trade-off, not a hidden flaw. The origin has to reach the model, because app-bound tools act on this turn's doorway and nobody else's. The line sits near the tail, after the whole vault. But consecutive turns from one doorway keep identical bytes only when nothing else moved, and two other things can move them. The vessel meta block at the very top names the brain, model and reasoning level this turn requested, so a turn on another vessel (Master Chief's whole point) or a changed model or reasoning level rewrites the first lines, which costs a prefix matcher nearly everything. And a turn that references an artifact appends that artifact to the system field for that turn alone. The price depends on the provider: on a whole-field matcher like Z.ai, a doorway switch forfeits that turn's entire system cache; on prefix matchers, how much survives depends on where their cache boundaries fall. Nobody has measured it in Master Chief yet. That is the honest state.

A cache contract holds only where something checks the bytes. The engine's tests guard the base prompt. Master Chief's origin line came in at the addendum seam, which the contract's own docstring never names. Decide what holds still, then make a test say so.

Code

Verbatim excerpts from the context engine: the contract in the build docstring, and the line where per-turn context joins the last user message·python

# from the build_system_prompt docstring
"""
PREFIX-CACHING CONTRACT (coop #117/#119, 2026-08-14): this string
must be byte-identical across every turn of a conversation.
Z.ai matches the whole ``system`` field exact (probe-verified:
one per-turn byte forfeits ~100K of cache); Anthropic/OpenAI/
Google match prefixes where the same rule holds a fortiori.
Per-turn content — the clock and the RAG retrieval — therefore
lives in ``build_turn_context`` and rides the current user
message (sent, never persisted). Do not add anything volatile
here; the prefix-stability tests fail loudly if you do.
"""

# the end of prepare_turn_messages
prepared[-1] = replace(
    prepared[-1],
    content=f"{turn_context}\n\n{prepared[-1].content}",
)
return prepared

Verbatim excerpts: the Master Chief route adds the origin line, and the chat route appends the addendum to the system prompt·python

# Master Chief route, on every Chief turn
payload["system_prompt_addendum"] = "\n\n".join(filter(None, [
    payload.get("system_prompt_addendum"), master_chief.CHIEF_ADDENDUM,
    "Current origin: " + json.dumps({k: context[k] for k in ("origin_surface", "origin_conversation_id")}),
]))

# chat route, after the conversation-stable prompt is built
if request.system_prompt_addendum and request.system_prompt_addendum.strip():
    system_prompt += "\n\n---\n\n" + request.system_prompt_addendum.strip()

A minimal sketch: the system bytes hold across a minute tick while the clock rides the message, and the stored rows stay clean (runs on Python 3.9+)·python

"""A minimal sketch: a system prompt that holds still, and a turn context that does not."""
from __future__ import annotations

import hashlib
from datetime import datetime, timedelta, timezone

KST = timezone(timedelta(hours=9))
VAULT = "Identity, shared rules, core memories: the same bytes on every turn."


def system_prompt() -> str:
    return VAULT  # nothing volatile may enter here


def turn_context(now: datetime, retrieved: str | None) -> str:
    parts = [f"[Current time: {now.astimezone(KST):%A, %B %d, %Y, %I:%M %p} KST]"]
    if retrieved:
        parts.append("## Retrieved Memory - Reference Material (background, NOT current task)\n\n" + retrieved)
    return "\n\n".join(parts)


def prepare_turn(history: list[dict], text: str, now: datetime, retrieved: str | None):
    sent = history + [{"role": "user", "content": f"{turn_context(now, retrieved)}\n\n{text}"}]
    stored = {"role": "user", "content": text}  # the row keeps Dad's words alone
    return sent, stored


history: list[dict] = []
start = datetime(2026, 1, 5, 21, 14, tzinfo=KST)
for minute, (text, retrieved) in enumerate([("Morning plan?", None), ("And the build?", "a note about builds")]):
    sent, stored = prepare_turn(history, text, start + timedelta(minutes=minute), retrieved)
    digest = hashlib.sha256(system_prompt().encode()).hexdigest()[:12]
    print(f"turn {minute + 1}: system sha256 {digest} | sent: {sent[-1]['content'].splitlines()[0]}")
    history.append(stored)
    history.append({"role": "assistant", "content": "(reply)"})

print("stored rows carry a clock:", any("Current time" in row["content"] for row in history))

Exercise

In an app of yours that calls a model API, log a sha256 of the system prompt on every request for a day of normal use. List every reason the hash changed. Move each per-request item (dates, user names, retrieved snippets, feature flags) out of the system prompt and onto the newest user message, keep the stored history free of it, and compare the cached-token counts the API reports before and after.

Hint

Anthropic reports cached tokens as cache_read_input_tokens in the usage block; OpenAI reports them under prompt_tokens_details.cached_tokens. Most providers only cache prompts above a minimum length, so test with a realistically long system prompt.

One Byte Is Enough

Where the Moving Parts Go

What It Bought, Measured

Two Five-Minute Clocks

The Exception, Told Plainly

Code

External links

Exercise

Progress

Comments 0