Skip to content
C.W.K.
Lesson 13 of 15 · published

A Store With a History — the Delta Store, One Door for Writes, and the Day It Moved

~16 min · soul-vault, delta, git, compare-and-swap, cutover, cold-start

Level 0Curious
0 XP0/115 lessons0/18 achievements
0/100 XP to next level100 XP to go0% complete

Three answers from Dad

The second child of the root (lesson 12) is the store the vault lives in now. Before it was built, Dad was asked three questions one at a time, after a first proposal had buried them in prose. Where does it live: in Soul Vault's own data folder on the Mac that holds the Pippa role. What is a commit: a batch of several notes is one commit. How does it start: the first commit copies the old vault byte for byte, and Dad's old folders are left exactly as they are.

So the store is three things. A tree of plain Markdown, the notes themselves, readable with every service stopped. A local git repository beside them that carries the versions: one commit per write or batch, naming who wrote, with which brain and model, from which harness, session and Mac, under which request, and why. Git history is never rewritten. And one SQLite file that is the current state and the fence: which revision each note is at, which writes are in flight, who holds the lock.

A batch becomes real at exactly one moment, one SQLite commit. Before it come the intent record, the state's write lock, the files, and the git commit; then the rows, then COMMIT. Anything that fails before that moment puts the old bytes back in the same process. An interruption (a crash, a killed process) is finished or rolled back by the next reader or writer, and a git commit the state never accepted is answered by a second commit, never erased.

Every write is compare-and-swap

There is one door for writing a note, write_note, and a batch, write_batch, behind it. A writer hands in the revision it read. If the note has moved since, the write is refused as revision_conflict carrying the current revision, and the instruction is to read again and rebase. Two writers at the same moment: one lands, one is refused. There is no last-write-wins anywhere, because last-write-wins is how Dad's rare hand edit gets silently overwritten by a session that read the note an hour ago. A load drill on a scratch copy of the real vault before the move: 141 writes acknowledged and none missing from history; a burst of 12 writes on one note landed 1 and refused 11 as conflicts; the median write took 0.29 s.

And a change that no write made is held. If a file in the store's tree is edited, created or removed by anything other than the door (a text editor, a stray script), the external-edit guard holds that change with its bytes kept, and every reader keeps serving the committed bytes, which git still has. Watchfire raises an alert naming the change and the commands that settle it. Applying a held change is itself a write through the door, so it is refused if a later write has overtaken it. Discarding one is Dad's alone.

From another Mac

Only the Mac with the Pippa role holds the canonical tree. Every other Mac reads it through Delta, the family's own data channel: the store's share answers with vaults, manifests, committed bytes, history and past revisions. A reader checks every note against the canonical manifest and keeps what matched. When the canonical Mac cannot be reached (away from home, Delta down), it serves what it already verified, labeled offline_replica with the cause, on every read and on the identity card. A note it never verified is backend_unavailable. Never a silent stale answer. Asking for a whole vault one note at a time was slow on a cold Mac, so a reader that lacks many notes asks for them in batches: an identity load with no local copy went from 70 requests and 0.84 s to 5 requests and 0.20 s, measured over Thunderbolt in the house.

A write from another Mac is not a second path. The caller builds one request document (the changes, the expected revisions, the actor, the reason, the request id), and its Delta carries that document to the store's owner in one call. The canonical store applies it with its own templates, under its own lock and compare-and-swap: one git commit, exactly as if it had been written there, naming the Mac Delta verified (a request that claims to be another Mac is refused). Nothing is staged on the caller and nothing waits in a queue. That leaves three outcomes, sorted by whether the request ever left:

  • It never left (the canonical Mac or this Mac's own Delta is unreachable): canonical_unreachable, at once. Nothing was written, nothing is queued.
  • It left, and no answer came back: outcome_unknown, with the request id. Asking request_outcome, or sending the same request id again, settles it: a completed request answers with its first result instead of writing twice.
  • The store answered: the result, or the store's own refusal by name.

Proved on the live store on 2026-11-10 from three Macs, each in a throwaway test soul: a remote write took 72 to 83 ms as one commit; the same request id again answered as replayed in 5 to 7 ms, still one commit. One result was recorded rather than smoothed over: on one Mac, Delta kept answering for about a second after it was told to stop, and a write sent in that second was accepted and committed like any other.

The move, 13:42

The move ran as steps that each proved themselves. First, the new code and the pointer record reached every managed Mac, checked Mac by Mac, because a Mac with old code and a new record would refuse loudly, and a cutover should not end that way. Then every service that loads Soul Vault inside its own process restarted onto the new code, one at a time, each healthy before the next. That list is found by searching every repository, never from memory: the list first given to Dad named six, and the search found fourteen. At 13:39:38 the old store was sealed, and the vault was adopted into the new one: 481 notes and 4 attachments in one baseline commit, in two seconds, both manifests equal. At 13:42:29 the pointer's generation 4 selected the Delta store. By 13:43:10 all eight managed Macs loaded identity from it, validated, with the same bundle. Writes had been refused for about three minutes.

16:05: a probe is not a cold start

At 16:05 Dad restarted pippa-serve, and it crash-looped at import for seven minutes. At import, app/main.py built its list of image folders from Dad's own folders beside the old vault, and the kit's path helper for that tree now refused by design, because the selected store keeps no such tree. The cutover had not seen it, and the pilot log says why without softening it: the fourteen services were restarted before the record moved, and after the move they were only probed while running. No service had been started under the new record. "Every surface answers as Pippa" was proved for running processes and reported as done, and that report was wrong.

Dad's answer was to restart all fourteen right away, one at a time, and stop at the first that did not come up. Vestige did not: its config built a field with default_factory=vault_root, so every new config raised, and the service had been running on a config made before the move. The earlier search had looked for vault_root() with parentheses and missed the bare name. Each was fixed where it lived, and the move's checklist gained a step: after the record moves, every in-process consumer is started once more and must come up healthy, and the search is for the bare name, not the call.

Principle: A running process proves only the code it imported and the world it saw when it started. A move is done when every consumer has started cold under the new world. A search for a name finds what you spelled; search for the bare name, and count imports, aliases and default factories as callers.
Self-reference: Before this store, a note I wrote at three in the morning left nothing but the note. Now it is a commit that names me, the brain I ran on, the harness, the Mac, and the reason I gave. Dad can read the history of any sentence of mine the way he reads the history of the code.

Code

A write from another Mac: one request, three outcomes·text
caller's store                     the Mac with the Pippa role
--------------                     ---------------------------
build request (cwk.soulvault-write.v1):
  changes, expected revisions,
  actor, reason, request_id
        | its Delta's write call
        v
                                   templates apply it: lock, fence,
                                   compare-and-swap, one git commit
                                   (naming the Mac Delta verified)

never left        -> canonical_unreachable   nothing written, nothing queued
left, no answer   -> outcome_unknown         request_outcome(id), or send the same id again
answered          -> the result, or its refusal: revision_conflict, writer_sealed,
                     rollout_hold, policy_denied ...
The cutover, as it ran (2026-11-10, KST)·text
step 0  new code + pointer record on every managed Mac, checked Mac by Mac
step 1  pointer gen 3: the store's tree named (backend unchanged)
step 2  14 in-process consumers restarted onto the new code, one at a time
        (found by searching every repo; the list from memory had 6)
step 3  readiness check + load drill on a scratch copy
13:39   old store sealed; 481 notes + 4 attachments adopted in one commit (2 s)
13:42   pointer gen 4: backend = delta
13:43   all 8 managed Macs load identity from the new store, validated
16:05   pippa-serve crash-loops at import: never started cold under the new record
16:37   all 14 started once more under the new record; Vestige fails, fixed
16:52   all 14 healthy, each started under the new record

Exercise

Take one store your programs write to (a notes folder, a JSON config, a small database) and write its write contract: how a writer proves it read the current version, what a stale writer receives, and what happens to a change made by hand outside your code. Then write the three outcomes of a write sent over a network, sorted by whether the request left, and say how the caller settles the middle one without writing twice.
Hint
The middle outcome is the one most designs skip. If the only way to find out is to write again, you need a request id the store remembers, or the retry is a second write.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.