Skip to content
C.W.K.
Lesson 01 of 05 · published

Verified Against the Manifest

~15 min · replication, manifests, integrity, offline-mode

Level 0Loose Note
0 XP0/40 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete

One Canonical Store, Many Readers

The Delta store lives on one Mac: whichever Mac holds the family's pippa role, the one that runs Pippa's services. Every other Mac (the workbench, the laptops, the build machines, Pippa's own home Mac, Mom's Mac, two Windows boxes through WSL) needs to load identity too, often several times an hour. A store that only one machine can read would put a network hop and a single point of failure in front of every session start.

So other Macs read through Delta, the family's own data channel between machines, and keep a local copy. The whole design question is how a Mac knows its copy is still the vault.

The Manifest Is the Truth About Bytes

The canonical store publishes, for each vault, a manifest: every note's id and the SHA-256 of its current bytes. A reader on another Mac asks its own Delta for that manifest, then for each note it needs:

  • If it holds a local copy whose hash matches the manifest, it serves that copy, labeled validated. No bytes cross the network.
  • If its copy is missing or behind, it fetches the canonical bytes, checks them against the manifest, keeps them, and serves them as validated.
  • If the canonical bytes themselves do not match the canonical manifest, that is integrity_error. Something is wrong at the source, and no answer is better than a wrong one.

A note written between the manifest and its read is read against a fresh manifest once before its bytes are called damaged, so an ordinary concurrent write does not masquerade as corruption.

When the Canonical Mac Is Away

Travel, a sleeping Mac, a restarting Delta: sometimes the manifest cannot be fetched. Dad's rule for this case is called D4 in the design, and it is carried through every layer:

  • A note this Mac verified earlier may still be served, labeled offline_replica with the cause. The label appears on the read, on the identity card, and in machine-readable output.
  • A note this Mac never verified is backend_unavailable. It is never fetched from anywhere else and never guessed. (If the manifest this Mac saved last time does not list the note at all, the answer is note_not_found instead, as it would be online.)
  • A write is canonical_unreachable, at once. Nothing is queued (lesson 4).
  • The next process that can reach the canonical Mac reads validated again. Nothing has to be reset.

The rare-path trial on 2026-11-10 exercised both halves. With only this Mac's Delta out of reach, the client asked the canonical Mac's Delta gateway directly over the tailnet and still read validated. With both out of reach for one process (its Delta client pointed where nothing listened, no service or network setting touched), a verified note was served as offline_replica with the cause on the read, the card and the JSON, a never-verified note was backend_unavailable, a write was canonical_unreachable with nothing queued, and the next process read validated.

A Soul That Leaves

One more rule keeps replicas honest: when a soul is removed on the canonical store, every other Mac drops its kept notes, manifest and rendered bundles for that vault the next time it reads a vault list that no longer names it. Not offline, and never on a list that lacks the shared vault, so a truncated answer cannot make a Mac throw away a soul that still exists.

Code

Serving from a verified copy (sketch of the reader's rule)·python
def serve(note_id, kept, saved, delta):   # saved: the last manifest this Mac kept
    try:
        manifest = delta.manifest(note_id.vault)              # {uri: sha256} from the canonical store
    except Unreachable as cause:
        if note_id.uri in kept:                               # verified earlier, by hash
            return kept[note_id.uri], f"offline_replica ({cause})"
        if saved is not None and note_id.uri not in saved:   # not a note, as far as we know
            raise VaultError("note_not_found", note_id.uri)
        raise VaultError("backend_unavailable", f"{note_id.uri}: no verified copy here; {cause}")
    want = manifest.get(note_id.uri)
    if want is None:
        raise VaultError("note_not_found", note_id.uri)
    data = kept.get(note_id.uri)
    if data is None or sha256(data) != want:
        data = delta.fetch(note_id)
        if sha256(data) != want:
            manifest = delta.manifest(note_id.vault)          # written meanwhile? ask once more
            if sha256(data) != manifest.get(note_id.uri):
                raise VaultError("integrity_error", f"{note_id.uri} does not match its manifest")
        kept[note_id.uri] = data
    return data, "validated"
Four answers a non-canonical Mac can give (D4)·text
canonical reachable, copy matches          -> bytes, validated (nothing transferred)
canonical reachable, copy behind/missing   -> canonical bytes, checked, kept, validated
canonical unreachable, note verified here  -> kept bytes, offline_replica (<cause>)
canonical unreachable, never verified      -> backend_unavailable
offline, absent from the saved manifest    -> note_not_found
(any write while unreachable)              -> canonical_unreachable, nothing queued

External links

Exercise

Make a read-mostly replica of a directory on a second machine that serves only what it verified: publish a manifest of hashes on the source, keep a verified copy on the replica, serve 'validated' when the manifest is reachable and matches, 'offline' with the cause when it is not and the copy was verified earlier, and a named refusal otherwise. Then unplug the network mid-run and watch every label.
Hint
Store the verification, not just the bytes: a file you copied but never checked against a manifest must not be served offline, because you cannot say what it is a copy of.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.