Intent, Lock, Files, Commit, Rows

Three Stores Pretending to Be One

The Delta implementation keeps a soul vault in three places at once, each doing what it is good at:

Plain Markdown files in one tree, so the notes open in any editor with every service stopped.
A local git repository over that tree, so every version of every note is kept and can be read back. It has no remote on purpose (Dad's answer on 2026-11-10); the family's hourly whole-data backup carries it.
One SQLite file, state.sqlite3, holding the current state: which notes exist at which revision, every revision ever recorded, every completed request, and the fence that a seal moves.

Three stores means three chances to disagree. A crash after the files changed but before git committed, or after git committed but before the rows were written, would leave a vault where the editor, the history and the state each tell a different story. The heart of the Delta child is one fixed order that makes that impossible.

The Order

Intent. Before touching anything, write an intent record to intents/<batch>.json, atomically: the batch id, the vault, the request id, and for every note both the old bytes and the new bytes (base64). This file is the plan and the undo log in one.
Lock. BEGIN IMMEDIATE on the state database: SQLite's write lock. From here until COMMIT, no seal can move the fence, so the "not sealed" check made right after holds for the whole publication.
Files. Write each new note atomically (write a temporary file, then rename) or unlink a deleted one, and sync the directory.
Commit. One git commit for the whole batch, whatever its size.
Rows. Insert a revision row per note, update the note rows, record the batch and the completed request.
COMMIT. In the source's words: "the batch is real from here". Then delete the intent and drop the rendered identity bundles that no longer match the vault, except those inside the ten-minute grace from Track 4.

SQLite's commit is the single moment of truth. Before it, nothing has happened as far as any reader of the state is concerned. After it, everything has.

When Something Fails Halfway

If anything raises before COMMIT, the same process rolls the transaction back and withdraws the batch at once, from the intent: every touched note goes back to its old bytes (or is unlinked if it did not exist before). If git had already committed the batch, the history is not rewritten. A second commit, "Withdraw batch ...: the store did not accept it", puts the old bytes back and leaves the first commit in place, naming no revision. If the process dies outright, the intent file is still on disk, and the next reader or writer finishes the job the same way before doing anything else.

A Commit That Says Who and Why

Every commit's subject is the write's reason when one was given (contract v5 made reason part of every write), and its trailers record the rest: Soul-Vault-Batch, Soul-Vault-Request, Soul-Vault-Actor (kind and name), and when known Soul-Vault-Brain, -Model, -Harness, -Session, -Mac, -Origin, -Approval, then Soul-Vault-Generation and one Soul-Vault-Note line per changed note, up to fifty; a bigger batch lists the first fifty and adds Soul-Vault-Notes-More with the count of the rest. The committer is "Soul Vault". Months from now, "who changed this line in my core, from which harness, and why?" is one git log away.

The history is append-only. The first commit to carry any notes (after the empty commit that set up the repository) was an import that copied all 481 notes and 4 attachments byte for byte as one adopt revision each, inventing no earlier history; an independent check matched all 481 SHA-256 hashes with zero mismatches.

Code

The publication order (a sketch of soulvault/backends/delta.py, names simplified)·python

intent = {"batch": batch, "vault": vault, "request_id": request_id,
          "ops": [{"path": op["path"], "op": op["op"], "old": b64(op["old"]), "new": b64(op["new"])}
                  for op in ops]}
atomic.write_atomic(intent_path, json.dumps(intent).encode())     # 1. intent
conn = self._db()
try:
    conn.execute("BEGIN IMMEDIATE")                                 # 2. lock
    self._refuse_when_sealed(conn)
    for op in ops:                                                  # 3. files
        target = tree / op["path"]
        if op["op"] == "put":
            atomic.write_atomic(target, op["new"])
        else:
            target.unlink()
    commit_id = self._commit(tree, pathspecs, message, actor)      # 4. one git commit
    for op in ops:                                                  # 5. rows
        conn.execute("INSERT INTO revisions(...) VALUES (...)", ...)
        conn.execute("INSERT INTO notes(...) ... ON CONFLICT(vault, path) DO UPDATE ...", ...)
    conn.execute("INSERT OR REPLACE INTO requests(...) VALUES (...)", ...)
    conn.execute("COMMIT")                                          # 6. the batch is real from here
except BaseException:
    conn.execute("ROLLBACK")
    self._withdraw(intent_path, intent)    # old bytes back now; a second commit if git already has it
    raise
intent_path.unlink()
self._invalidate_bundles()                 # renders older than the ten-minute grace

What a write's commit carries (shape; values invented)·text

add the evening's correction to the travel note

Soul-Vault-Batch: 5e0c...
Soul-Vault-Request: note-edit:91a4...
Soul-Vault-Actor: soul:pippa
Soul-Vault-Brain: claude
Soul-Vault-Harness: claude-code
Soul-Vault-Mac: <fleet label>
Soul-Vault-Generation: 5
Soul-Vault-Note: put pippa/memory/<note>.md

Exercise

Implement a two-store write for something of yours (files plus a SQLite index, or files plus git): write an intent with old and new bytes first, take the database's write lock, change the files, then the second store, then the rows, then commit. Make any exception before the commit restore the old bytes from the intent, and add a startup step that finishes or withdraws any intent it finds. Kill the process at each step and check that every restart converges.

Hint

Test the crash at every boundary, not only the middle. The interesting failures are a crash after the files changed but before the commit, and a crash after the commit but before the intent was deleted. The second must be harmless.