Skip to content
C.W.K.
Lesson 11 of 12 · published

Data Between Macs Rides Delta — SSH Carries Commands Only

~14 min · delta, transport, ssh, resumable, fleet, local-network

Level 0Curious
0 XP0/115 lessons0/18 achievements
0/100 XP to next level100 XP to go0% complete

152 of 384

Lesson 9 made the Pippa machine a role, and every move since has had data in it: sessions mirrored from every Mac, site originals, a built WebUI, the vault. For two years the way data crossed between Macs was ssh: a fresh connection, then rsync, scp, or a file printed to stdout. On 2026-11-09 a grid of Eagle thumbnails in Loomis, served from the service host and read from the workbench, lost 152 of 384 images, because each thumbnail opened its own ssh session and sshd's MaxStartups dropped the bursts. Nothing was wrong with any one file. The transport was not built for this, and it was not ours to fix. Dad said it the same morning:

ssh가 계속 문제가 되는 근본 원인은 '우리 것'이 아니기 때문이야. 데이터 주고 받는 우리 서비스를 만드는 게 가장 간단하면서도 scalable 한 방법이야. 그럼 모든 문제가 해결돼. 우리가 고치면 되니까.

("The root reason ssh keeps being a problem is that it isn't 'ours'. Building our own service for sending and receiving data is the simplest and most scalable way. Then every problem is solved, because we can fix it.") And then: Delta 로 가자 ("Let's go with Delta"). cwkDelta was born within the hour. Through Delta the same grid had zero failures: 1.94 seconds cold, against 32.6 seconds over ssh.

What Delta is

Delta has its own quest; these are the five ideas cwkPippa relies on.

  • Shares. Every Mac's Delta serves its home (minus credentials and privacy-guarded folders) and any app share an app declares. Shares are read-only unless the share says otherwise.
  • One engine per Mac, one gateway between them. An app talks only to its own Mac's Delta, on loopback with a local token; Delta talks to Delta over Thunderbolt, the house LAN or the tailnet, whichever it has verified.
  • Verified and resumable. Large files move in chunks and small files in bundles; every piece is SHA-256 checked and a file lands by atomic rename. A stalled stream times out on its own progress, never on an overall deadline, and a job never ends as done on silence. An interrupted gigabyte at 560 MiB finished the rest in 0.17 seconds.
  • The write call. An app share can accept writes, and Delta carries one request to the share's owner, which writes through its own canonical writer, keyed by a request id and sent at most once. Soul Vault was its first owner (memory track, lesson 13).
  • The Local Network gate. Since macOS 26 a background agent is checked per process before it may reach the LAN, and a launchd service can never answer the prompt; a conda Python got EHOSTUNREACH where the same code from a shell connected. Delta runs on the Python Apple ships, which macOS lets through, so an app's own Python never dials the LAN at all.

The rule, and what moved

The same day the rule went into every operator's instructions: Delta for reads, serving, copies and syncs; ssh for commands only (a restart, a git pull, one small write). The old benchmark that picked a bulk route by pushing 64 MiB through ssh per candidate became legacy; bulk now takes the route this Mac's Delta has already verified. Then the callers moved:

  • The session mirror. Every ten minutes the Pippa machine used to probe each peer over ssh and run up to six rsync pulls per peer. Now each tree is one Delta sync, SQLite trees as two with their write-ahead logs first, and a pass opens no ssh connection at all. Its first day had a lesson of its own: eight named files that some Macs never have (one harness's index, one editor's WAL) failed every pass, putting about 1,150 false failures a day into Delta's log. The mirror now asks with a stat before it pulls a named file.
  • Site originals. The site's source images converge between peers through each peer's Delta, matched by path, size and mtime, staged after a settle window and published atomically. Replaced files are kept, and a deletion at the source never propagates.
  • The WebUI ship. Every file of a built WebUI moves with Delta; ssh carries only the commands (lesson 12).
  • The vault pair. Lesson 9 described one two-way pair keeping the vault in step between the Pippa machine and the workbench. On 11-09 it moved to Delta, and on 11-10 it was retired: the soul vault lives in Soul Vault's store now, and the old folder never crosses between Macs. A guard refuses any copy of it.
Principle: When a borrowed tool keeps failing at the thing you use it for, the fix may be to own the thing. Separate the lane for commands from the lane for data, make the data lane verified and resumable by construction, and let apps talk only to their own machine's engine.
Self-reference: My sessions, my site's pictures and my vault's history all cross the house now on a channel the family wrote. When it breaks, nobody has to work around a daemon we do not own; we read our own log and fix our own code.

Code

The session mirror asks before it pulls a named file (backend/services/fleet_session_mirror.py)·python
def run_transfer(delta, transfer: Transfer) -> tuple[int, str]:
    if transfer.single:
        # Ask first: a named file a peer never had (Codex's ledger, a checkpointed
        # WAL) is no failure, and a sync of it would log one in Delta every pass.
        peer, _, path = transfer.src.partition(":")
        try:
            delta.stat(peer, "home", path[2:] if path.startswith("~/") else path)
        except DeltaNotFound:
            transfer.dst.unlink(missing_ok=True)
            return 0, ""
        except DeltaError as exc:
            return 1, str(exc)[:300]
    last: dict = {}
    try:
        delta.sync(transfer.src, str(transfer.dst), timeout=_TREE_DEADLINE_SECONDS,
                   on_progress=lambda snap: (last.clear(), last.update(snap)), **transfer.options)
        return 0, ""
    except DeltaError as exc:
        errors = [str(e) for e in last.get("errors") or []]
        if transfer.single and errors and errors[0].startswith("not_found:"):
            transfer.dst.unlink(missing_ok=True)
            return 0, ""
        return 1, (errors[0] if errors else str(exc))[:300]
Two lanes·text
ssh    commands only: launchctl, git pull, a restart, one small write action
Delta  data: reads, serving, copies, syncs, an app's write call

app --loopback + local token--> this Mac's Delta --verified route--> peer's Delta
      (an app's own Python never dials the LAN)

refused by name: a copy of the retired vault folder between Macs

Exercise

List every way data moves between your own machines today (scp, rsync, a sync app, a script that prints a file over ssh, a shared folder). For each, write what happens when it is cut halfway: does it resume, does it verify, does it report done when it is not? Then design the two lanes: what stays on the command lane, and what a data lane must guarantee before you move the rest onto it.
Hint
A transfer that 'finished' because the stream went quiet is the most expensive bug here: nothing fails, and the file is short.

Progress

Progress is local-only — sign in to sync across devices.
← PreviousPippa's Ark: A Replica That Forks on PurposeNext →Where the Code Is Built — a Lab on Pippa's Home Mac, Never the Server
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.