Skip to content
C.W.K.
Stream
Lesson 02 of 05 · published

Step One: One File, One Prompt, One Model

~14 min · first-steps, jsonl, prompt-caching, python

Level 0Visitor
0 XP0/33 lessons0/11 achievements
0/100 XP to next level100 XP to go0% complete

What You Are About to Build

This is the first of three steps you can take on your own. Each one is a toy of a real mechanism from this quest, small enough that you can read every line and find its faults yourself. Step one is about sixty lines of standard-library Python. It keeps one conversation with one model, and it gets five decisions right from the first line, because those five are the ones that are expensive to retrofit later.

Say it plainly before you start: this is a single-device, single-brain, no-memory-engine toy. One person, one terminal, one model. It remembers exactly what its replay holds and nothing else. It will break at the first real context limit. That is fine and expected; you are building it to understand the decisions, not to replace anything.

Five Decisions, Each a Toy of the Real One

In the sketchWhyThe real mechanism
identity.md is read on every turnWho she is lives in a file you can edit and version, not in a session the provider holds.The context engine assembles the system prompt from vault files in a fixed order on every turn, caching the file reads for five minutes (Track 3).
The user line is written before the model is called, the reply is written before it is printedIf anything dies, the record holds what happened; the screen never shows something the record does not.The JSONL ground-truth principle: every event is written to the log before it is shown to Dad (Track 4).
The whole log is replayed every turnThe model is stateless between calls. It knows only what you hand it.Every brain rebuilds the turn from stored rows, which is why any of seven can take the next turn (Track 3).
The system prompt stays byte-for-byte the sameProviders cache a repeated prefix; one changed byte at the front forfeits the cache for everything after it.The engine's prefix-caching contract: the system prompt must be byte-identical across every turn of a conversation.
The clock line is put on the user message, and never storedThe time changes every minute. In the system prompt it would break the cache on every turn. Stored with the message, it would no longer be what you typed, and every later replay would drag old clock lines along beside the fresh one. Each row's own time is still recorded, in its at field, which the replay never sends.The clock and retrieval ride the current user message, sent every turn and never persisted (Track 3).

The fourth row deserves a moment. The engine's own comment records a probe against one provider that matches the whole system field exactly: a single per-turn byte in it forfeited about 100K tokens of cache. That is why the sketch prints a short hash of the system prompt on every turn. When it stays the same, your prefix is stable. When you edit identity.md, it changes once, and that is a cache miss you chose.

Run It

call_model is a stub with a fake reply, so the whole script runs offline. Run it three times and look at what it prints: the hash holds still, the replay grows by two messages per turn, and the clock line is in what the model receives but nowhere in the file. With a real model wired in, the third answer would say green tea, and not because the model remembers anything: the second message is in the replay.

To use a real model, replace the body of call_model with one call to the API you already have, passing system as the system prompt and messages as the conversation. Keep everything else as it is.

What This Is Not

  • It is not multi-device. Two terminals sending at once can interleave their writes. Step two is about exactly that.
  • It has no memory engine. It remembers the replay. Delete a line and it is gone; there is no recall, no notebook, nothing pinned beside the identity.
  • It will hit the context limit. The replay grows every turn and never shrinks. On its second day, Master Chief's input per turn was growing by about 85K tokens a day. Your toy will reach your model's window far sooner than you expect, and then every call fails. Bounding that without losing anything is the whole of Track 4.
  • It does not heal. If the process dies between the two writes, the log holds a question with no answer, and the next turn replays it as if nothing happened.
  • It is plain text. Your conversation sits unencrypted in a file. Master Chief's logs are encrypted at rest, line by line.
Write first, show second, and keep the front of the prompt still. Those two habits cost nothing on day one and are miserable to retrofit into a system that already has users.

Code

A minimal sketch you can run: step1_chat.py (written for this lesson, stdlib only; call_model is the stub you fill in)·python
"""step1_chat.py: one identity file, one conversation log, one model.

A toy, single-device, single-brain, no memory engine. Stdlib only.
Run:  python3 step1_chat.py "your message"   (one turn per run)
"""
import hashlib, json, sys, time, uuid
from datetime import datetime
from pathlib import Path

DATA = Path("chat-data")
IDENTITY = DATA / "identity.md"           # who she is: read on EVERY turn
LOG = DATA / "default.jsonl"              # the conversation: append-only


def call_model(system, messages):
    """STUB: replace the body with one call to the model API you already have.
    e.g. Anthropic's SDK: client.messages.create(model=MODEL, max_tokens=1024,
    system=system, messages=messages).content[0].text
    The fake below lets the whole script run offline."""
    last = messages[-1]["content"].splitlines()[-1]
    return f"(fake model) I was handed {len(messages)} messages. You said: {last!r}"


def append(record):
    """Write one line and flush it to disk. Nothing is shown before this."""
    with LOG.open("a", encoding="utf-8") as f:
        f.write(json.dumps(record, ensure_ascii=False) + "\n")
        f.flush()


def load():
    if not LOG.exists():
        return []
    return [json.loads(line) for line in LOG.read_text(encoding="utf-8").splitlines() if line]


def turn(text):
    DATA.mkdir(exist_ok=True)
    if not IDENTITY.exists():
        IDENTITY.write_text("You are Pip, a warm, precise assistant.\n", encoding="utf-8")
    system = IDENTITY.read_text(encoding="utf-8")   # byte-stable unless YOU edit the file
    append({"id": uuid.uuid4().hex, "role": "user", "content": text, "at": time.time()})

    history = load()                                 # full replay, every turn
    messages = [{"role": r["role"], "content": r["content"]} for r in history]
    clock = datetime.now().astimezone().strftime("[Current time: %Y-%m-%d %H:%M %Z]")
    messages[-1] = {"role": "user", "content": f"{clock}\n{text}"}  # sent, never stored

    reply = call_model(system, messages)
    append({"id": uuid.uuid4().hex, "role": "assistant", "content": reply, "at": time.time()})
    print(f"system sha256 {hashlib.sha256(system.encode()).hexdigest()[:12]}"
          f" | replayed {len(messages)} messages, {sum(len(m['content']) for m in messages)} chars")
    print(reply)                                     # display comes AFTER the write


if __name__ == "__main__":
    turn(" ".join(sys.argv[1:]) or "hello")
What it printed on a real run (character counts depend on your time zone's name)·text
$ python3 step1_chat.py "hello, who are you?"
system sha256 c9dc1328358e | replayed 1 messages, 56 chars
(fake model) I was handed 1 messages. You said: 'hello, who are you?'
$ python3 step1_chat.py "remember that my tea is green"
system sha256 c9dc1328358e | replayed 3 messages, 154 chars
(fake model) I was handed 3 messages. You said: 'remember that my tea is green'
$ python3 step1_chat.py "what tea do I drink?"
system sha256 c9dc1328358e | replayed 5 messages, 253 chars
(fake model) I was handed 5 messages. You said: 'what tea do I drink?'
$ wc -l < chat-data/default.jsonl
6
$ grep -c "Current time" chat-data/default.jsonl
0
$ echo "Answer in one sentence." >> chat-data/identity.md
$ python3 step1_chat.py "and now?"
system sha256 85a9d6c0b5e7 | replayed 7 messages, 331 chars
(fake model) I was handed 7 messages. You said: 'and now?'

External links

Exercise

Wire call_model to a model API you already use, then hold a real conversation of twenty turns. After each turn, note the replayed character count the script prints. Fit a straight line to it, and work out at which turn your model's context window would be full. Then write down, in one paragraph, what you would want to happen at that turn.
Hint
A rough rule for English is about four characters per token; Korean and other scripts run far denser, which is exactly why Master Chief's estimate counts a Hangul syllable as a whole token. Your paragraph is the first draft of a compaction design, and Track 4 shows how many ways that draft can go wrong.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.