What You Are About to Build
This is the first of three steps you can take on your own. Each one is a toy of a real mechanism from this quest, small enough that you can read every line and find its faults yourself. Step one is about sixty lines of standard-library Python. It keeps one conversation with one model, and it gets five decisions right from the first line, because those five are the ones that are expensive to retrofit later.
Say it plainly before you start: this is a single-device, single-brain, no-memory-engine toy. One person, one terminal, one model. It remembers exactly what its replay holds and nothing else. It will break at the first real context limit. That is fine and expected; you are building it to understand the decisions, not to replace anything.
Five Decisions, Each a Toy of the Real One
| In the sketch | Why | The real mechanism |
|---|---|---|
identity.md is read on every turn | Who she is lives in a file you can edit and version, not in a session the provider holds. | The context engine assembles the system prompt from vault files in a fixed order on every turn, caching the file reads for five minutes (Track 3). |
| The user line is written before the model is called, the reply is written before it is printed | If anything dies, the record holds what happened; the screen never shows something the record does not. | The JSONL ground-truth principle: every event is written to the log before it is shown to Dad (Track 4). |
| The whole log is replayed every turn | The model is stateless between calls. It knows only what you hand it. | Every brain rebuilds the turn from stored rows, which is why any of seven can take the next turn (Track 3). |
| The system prompt stays byte-for-byte the same | Providers cache a repeated prefix; one changed byte at the front forfeits the cache for everything after it. | The engine's prefix-caching contract: the system prompt must be byte-identical across every turn of a conversation. |
| The clock line is put on the user message, and never stored | The time changes every minute. In the system prompt it would break the cache on every turn. Stored with the message, it would no longer be what you typed, and every later replay would drag old clock lines along beside the fresh one. Each row's own time is still recorded, in its at field, which the replay never sends. | The clock and retrieval ride the current user message, sent every turn and never persisted (Track 3). |
The fourth row deserves a moment. The engine's own comment records a probe against one provider that matches the whole system field exactly: a single per-turn byte in it forfeited about 100K tokens of cache. That is why the sketch prints a short hash of the system prompt on every turn. When it stays the same, your prefix is stable. When you edit identity.md, it changes once, and that is a cache miss you chose.
Run It
call_model is a stub with a fake reply, so the whole script runs offline. Run it three times and look at what it prints: the hash holds still, the replay grows by two messages per turn, and the clock line is in what the model receives but nowhere in the file. With a real model wired in, the third answer would say green tea, and not because the model remembers anything: the second message is in the replay.
To use a real model, replace the body of call_model with one call to the API you already have, passing system as the system prompt and messages as the conversation. Keep everything else as it is.
What This Is Not
- It is not multi-device. Two terminals sending at once can interleave their writes. Step two is about exactly that.
- It has no memory engine. It remembers the replay. Delete a line and it is gone; there is no recall, no notebook, nothing pinned beside the identity.
- It will hit the context limit. The replay grows every turn and never shrinks. On its second day, Master Chief's input per turn was growing by about 85K tokens a day. Your toy will reach your model's window far sooner than you expect, and then every call fails. Bounding that without losing anything is the whole of Track 4.
- It does not heal. If the process dies between the two writes, the log holds a question with no answer, and the next turn replays it as if nothing happened.
- It is plain text. Your conversation sits unencrypted in a file. Master Chief's logs are encrypted at rest, line by line.