One Request for a Vault

Correct, and Slow

The verified-read rule from the last lesson is correct note by note. On a Mac that already holds a copy, it is also cheap: the manifest comes over, every hash matches, and nothing else moves. The problem is the Mac that holds no copy: a fresh state folder, a machine that never read a particular vault, a sandbox that cannot keep anything between processes. For that Mac, "note by note" means one network request per note.

Measured on the workbench against the live store on 2026-11-10, in the same house (a Thunderbolt link), reading every note of every vault with no local copy took 495 requests and 5.5 seconds, of which 480 one-note requests took 4.4 seconds. An identity load alone, 67 documents with no copy, took 70 requests and 0.84 seconds. Fine in the house; painful on a hotel link, where every one of those requests is a round trip.

Ask for a Vault, Not a Note

Coop todo 1110-41 added one question to the store's share: notes?vault=, which answers with the notes of one vault, either all of them or a named list (paths, a JSON list, a few thousand bytes of names per request). Every note in the answer is still checked against the manifest the reader holds, and kept only if it matches. What changed is how many questions it takes. The reader picks by how much it lacks:

Four notes or fewer missing: read them one by one, as before. A single note someone just wrote does not justify a batch.
More than four missing: ask in batches, naming exactly the notes it lacks, so a handful of notes written elsewhere never drags a whole vault across a slow link.
Half the vault or more missing: ask for all of it in one answer. That answer is the same for every reader at that vault version, so the reader's own Delta keeps it for the next reader on that Mac.

A reader that cannot keep its copy (a read-only sandbox) holds what it fetched for the life of its process. A canonical Mac whose share does not know the new question yet is simply read note by note, as before, so old and new can coexist during a rollout.

The Numbers After

Same Mac, same store, same day:

Every note of every vault, no copy: 495 requests in 5.5 s before, 33 requests in 1.1 s after (16 batch requests took 0.48 s).
Identity load, no copy: 70 requests in 0.84 s before, 5 requests in 0.20 s after.
Identity load, copy in place: 3 requests, 0.09 s before and 0.04 s after.
A second pass in the same process: 0 requests either way.

The store held 480 notes in 14 vaults (3.75 MB) at the first measurement and 496 in 16 at the second. One honest cost: a batch answer carries its bytes as base64, 5.1 MB on the wire for 3.8 MB of notes. And one honest gap: away from home was not measured. The architecture document says so in as many words, and the only defensible claim is the qualitative one: every request saved is one round trip of whatever link you are on.

Where the Old Numbers Came From

An earlier review had reported 2,123 one-note requests per pass. That turned out to be the retired folder implementation proving its copy note by note through its own share (174,177 such requests in that share's log before the move). The new store's readers verify a kept copy against the manifest without asking for each note, so after the move the one-note requests only ever came from a Mac with no copy, which is exactly the case this lesson fixed. Measure before you optimize, and make sure the thing you measured is the thing still running.

Code

Choosing how to read what is missing (sketch of the thresholds)·python

def fetch_missing(vault, manifest, kept, share):
    missing = [uri for uri, sha in manifest.items()
               if uri not in kept or sha256(kept[uri]) != sha]   # an empty note is a note
    if not missing:
        return {}
    if len(missing) <= 4:                                  # a note or two: one by one
        got = {uri: share.note(uri) for uri in missing}
    elif len(missing) * 2 >= len(manifest):                # half or more: the whole vault, once
        got = share.notes(vault=vault)
    else:                                                  # some: name exactly what is lacking
        got = {}
        for chunk in chunks(missing, by_bytes=4000):
            got.update(share.notes(vault=vault, paths=chunk))
    return {uri: data for uri, data in got.items()
            if uri in missing and sha256(data) == manifest[uri]}   # keep only what verifies

Measured on the workbench, same house, 2026-11-10·text

read                                         before                  after
-------------------------------------------  ----------------------  -----------------------
every note of every vault, no copy here      495 requests, 5.5 s     33 requests, 1.1 s
identity load (67 documents), no copy        70 requests, 0.84 s     5 requests, 0.20 s
identity load, copy in place                 3 requests, 0.09 s      3 requests, 0.04 s
a second pass in the same process            0 requests              0 requests

not measured: away from home (each saved request = one round trip of that link)

Exercise

Take a client that fetches many small items one request at a time. Measure it cold (nothing cached) and warm. Add a batch endpoint that returns a named list or everything, choose between per-item, named-batch and everything by how many items are missing, and keep only items that verify against a manifest. Measure again, and write down which links you did and did not measure.

Hint

Put the thresholds in one function with a comment explaining each number, and measure on the slowest link you can reach. The difference between per-item and batched grows with every millisecond of round-trip time.