Lesson 04 of 05 · published

Moving an App off SSH

~14 min · kit_delta, migration, client-library, mirroring, two-way-sync, error-handling

Level 0Loose File

0 XP0/43 lessons0/12 achievements

0/100 XP to next level100 XP to go0% complete

The Client Every App Uses

An app never talks to another Mac. It talks to its own Mac's Delta, on loopback, through one small client the family vendors into every app that needs it: kit_delta.py from cwkSiblingKit, standard library only and Python 3.9-compatible, because scheduled callers run Apple's python too. Constructing it takes one argument, the app's name: Delta(app="cwkLantern"). That name travels as X-Delta-App on every request, and a share's apps list is held against it (Track 2). The client reads this Mac's local.token from Delta's state folder (again at most once a minute, so a regenerated token is picked up) and finds the engine port in the family ledger. No address, port or token is ever written in the app.

The client's verbs map one to one onto what the earlier tracks taught:

Verb	What it asks	Track
`read(peer, share, path, expect=…)`	one file's bytes through the cache; the `Blob` says `hit`, `revalidated`, `miss`, `stale` or `pass` and which route it took	5
`stat`, `list`, `api`	metadata, a listing (one JSON line per entry), a child share's read operation	2
`copy`, `sync`	a job inside this Mac's Delta, polled to the end; `unique`, `include`, `exclude`, `delete`, `checksum` pass through	6
`pair`, `pair_state`	a named two-way pair's run (init, sync, verify) and what this Mac holds for it	7
`call(…, request_id=…)`	a write the share's owner carries out, sent at most once	7

Two conveniences are worth knowing because they are deliberately not everywhere. When this Mac's Delta is down, a read falls back to the peer's gateway directly over the tailnet: no cache, still no SSH, and the Blob's route says tailnet-direct so nobody mistakes it for the normal path. A write call never does, because the local Delta and its token are what vouch for this Mac (Track 7). And a transfer is polled with quick looks first (20, 50, 100, 200, 400, 700 ms) and then once a second: a one-file drop lands in tens of milliseconds, and a flat one-second poll made every such call a second long. A job that ends in anything but done raises a DeltaError whose cause is transfer_<state>.

Case One: Lantern's Mirror and Its Drop Folder

Lantern, the family's corpus engine, indexes Dad's writing. The writing lives on the workbench; the engine runs on the service host. Before 2026-11-09 the engine pulled its mirror with rsync over ssh, and every upload from Lantern's console opened its own ssh session carrying the file's bytes into a remote shell. The move to Delta (Lantern commit 07a8d9e, the day Delta was born) changed three things, and each is a pattern:

The mirror is a one-way sync from the role's Mac, with delete=True and the corpus's extension globs as include patterns. Delta's rule that a run which met an unreadable folder deletes nothing (Track 6) is exactly what a mirror needs: an interrupted or half-blind pull never leaves the mirror with fewer files.
The drop folder is a create-only copy with unique=True: the upload takes its name or the first free name (2) … name (50), never replacing a file, and the job reports which name it took.
Failures keep their causes. Because every Delta error arrives as <cause>: <detail>, Lantern can sort them: unavailable or timeout means the workbench never answered (asleep, off the network), so an upload is kept in an outbox and delivered later; not_found is "missing on that Mac"; forbidden or unreadable is "that Mac refused". With rsync over ssh all three were one exit code.

One detail had to change with it. Delta serves the home folder, never /tmp, so a console upload is staged under the engine's data root, where this Mac's Delta can read it, before the copy is asked for. A channel with refusals is a channel you have to stage for.

Case Two: Claude Code's Memory, Kept by a Pair

Claude Code writes what a session learned into ~/.claude/projects/<key>/memory/, one Markdown note per fact plus an index. Sessions on two workbench Macs wrote there independently, and on 2026-11-10 the two Macs disagreed on 98 notes across 30 project keys. The only path between them was a skill run by hand that copied over ssh. cwkPippa's claude-memory-pair.py replaced it with one Delta named pair per project key (a pair names two folders, and those folders sit among gigabytes of session logs that must not ride along). Adopting a key is careful in the way Track 7 taught: a note only one Mac has crosses first by copy; then pair(mode="init") proves both trees equal; a note that differs on both sides is left for a hand merge rather than guessed. The one exception is the index, which is one line per note, so the script unions the two indexes instead of calling it a conflict.

The Recipe

Find the bytes. List every ssh, scp and rsync in the app and split them: a command (launchctl, git pull, a restart, one small write action) stays on SSH; anything that carries file contents or reads a remote command's output as data moves. The scanner below does the first pass.
Name the role, not the Mac. @workbench, @pippa: the ledger says which Mac plays it today.
Pick the verb by intent. One file inside a request: read with expect. A mirror: sync. A drop: copy(unique=True). Two-way: pair. A write the owner must make itself: call.
Keep the causes apart. "Could not reach" is not "missing" and not "refused"; each needs a different answer from the app.
Never fall back to SSH. A fallback to the path you are leaving keeps its failure mode alive under the new one.

Code

Lantern after the move: one-way mirror, create-only drop, causes kept apart (trimmed excerpt)·python

# cwkLantern backend/services/source_mirror.py (trimmed excerpt)
from kit_delta import Delta, DeltaError, DeltaUnavailable

APP = "cwkLantern"
# A job error whose cause means the other Mac never answered (Delta names
# every failure "<cause>: <detail>"); anything else is that Mac's own answer.
_SILENT_CAUSES = ("unavailable:", "timeout:")


def classify(errors, message, machine):
    """(the precise reason a Delta job failed, whether the Mac never answered)."""
    lines = [e for e in errors if e] or ([message] if message else [])
    first = lines[0] if lines else "no detail"
    if any(line.startswith(_SILENT_CAUSES) for line in lines):
        silent = next(line for line in lines if line.startswith(_SILENT_CAUSES))
        return f"{machine} did not answer (asleep or off the network?): {silent[:200]}", True
    if first.startswith("not_found:") or ": not_found:" in first:
        return f"missing on {machine}: {first[:200]}", False
    if first.startswith(("forbidden:", "unreadable:", "unauthorized:")) or ": forbidden:" in first:
        return f"{machine} refused the read: {first[:200]}", False
    return f"Delta: {first[:200]}", False


# in pull_corpus(): the mirror is a one-way sync FROM the role's Mac
if kind == "root":
    # One-way mirror: deletions only after every folder was read, so an
    # interrupted or half-blind pull never leaves the mirror with fewer files.
    snap = delta.sync(src, str(dest), delete=True, include=include,
                      timeout=PULL_TIMEOUT_S, on_progress=watch)

# push_new_file(): a console upload lands as a NEW file in the role's inbox
snap = delta_client().copy(str(absolute), dest, unique=True, timeout=PUSH_TIMEOUT_S,
                           on_progress=watch)

Run it: find the places a codebase moves data, not commands, over SSH·python

# find_ssh_data.py: list the places a codebase moves DATA over SSH.
# Usage: python3 find_ssh_data.py <repo>
# Commands over SSH are fine (launchctl, git pull, a restart). Bytes are not.
import re
import sys
from pathlib import Path

CARRIES_DATA = [
    (re.compile(r"\b(scp|rsync)\b"), "copies files over ssh"),
    (re.compile(r"\bssh\b.*\b(cat|base64|tar|head|tail)\b"), "moves file bytes through a remote shell"),
    (re.compile(r"\bssh\b.*(>|<)\s*\S"), "pipes data through ssh's stdin/stdout"),
    (re.compile(r"""check_output\(\s*\[\s*["']ssh["']"""), "captures a remote command's output as data"),
]
COMMAND_ONLY = re.compile(r"\bssh\b.*\b(launchctl|git (pull|fetch)|kickstart|systemctl)\b")


def scan(root):
    hits = []
    for path in sorted(Path(root).rglob("*")):
        if path.suffix not in (".py", ".sh", ".zsh", ".bash") or ".git" in path.parts:
            continue
        try:
            lines = path.read_text(errors="replace").splitlines()
        except OSError:
            continue
        for number, line in enumerate(lines, 1):
            if line.lstrip().startswith("#") or COMMAND_ONLY.search(line):
                continue
            for pattern, why in CARRIES_DATA:
                if pattern.search(line):
                    hits.append((path, number, why, line.strip()[:100]))
                    break
    return hits


if __name__ == "__main__":
    root = sys.argv[1] if len(sys.argv) > 1 else "."
    found = scan(root)
    for path, number, why, text in found:
        print(f"{path}:{number}: {why}\n    {text}")
    print(f"{len(found)} place(s) move data over ssh in {root}")

External links

Exercise

Run the scanner on a repository of your own that talks to a second machine. For every hit, write one line: what bytes move, which verb would carry them (read, sync, copy with unique, pair or an owner-side call), and which three failure causes the caller must tell apart. Then check the lines the scanner did NOT flag that still call ssh: confirm each is a command, not data.

Hint

The scanner is a first pass, not a verdict. A check_output(["ssh", …]) that reads a one-word status is a command; one that reads a file's contents is data. Anything piped through < or > across ssh, or base64 on stdout, is data however small it looks today.