Stops Before Any Change

A Stop Is Not a Failure Halfway Through

Most errors in a transfer happen to one file: it could not be read, a chunk stalled, a destination changed under the plan. Track 6 handles those by naming the file and letting the rest land. A two-way sync has a second class of problem, where the whole plan is untrustworthy. Carrying out any part of a plan you cannot trust is worse than carrying out none, so pair.py has a separate exception for it: Stop, documented as "a reason no retry clears by itself; nothing was changed."

Every stop is raised before the first keep, the first transfer and the first delete. The job ends failed, its cause is the stop's name, its message says what to do, and both trees are exactly as they were. Through the kit (kit_delta's pair()) every stop arrives as the same DeltaError, cause transfer_failed, with the stop's message as its text. The stop's own name is only in the job snapshot, so an app that must tell big_delete from clock_skew passes on_progress and reads cause from the last snapshot it was handed.

The Stops, One by One

Cause	When	Why it stops instead of guessing
`clock_skew`	the two clocks differ by more than `max_skew_s` (default 60 s)	newer-wins compares two Macs' mtimes; with a skewed clock "newer" picks the wrong side
`clock_unread`	the peer's `/v1/health` carries no `time` (a Delta too old to report its clock)	a skew that cannot be measured cannot be bounded either
`archive_missing`	no archive for this pair	a fresh archive over unequal trees resurrects deletions (lesson 1)
`archive_unreadable`	an archive that does not parse, or is not one this Delta wrote	a corrupt memory is no memory
`pair_changed`	the run names other roots or other excludes than the archive	one tree's history would be read against another tree
`big_delete`	a whole tree, or more deletions one way than `max(20, half the archive)`, gone on one side	an unmounted disk or a moved folder looks exactly like "delete everything"
`unreadable`	a folder in either tree would not open	a folder that cannot be listed is not an empty folder
`root_missing`	a pair's root does not exist on one side	same as a whole-tree delete, one step earlier
`trees_differ`	`init` or `verify` found a difference	an archive must start from proven equality

The clock is measured, not assumed. A peer's /v1/health carries its time; the run records when it asked and when the answer came, and takes the peer's time minus the midpoint, which cancels half the round trip. A follow-up commit the same day made every mode report the skew while only sync refuses on it: init and verify compare content and never pick a winner by time, so they have no reason to stop, but the number is still worth seeing.

big_delete is the stop that protects against the most ordinary accident. Someone renames the pair's folder, an external disk is not mounted, a restore is half done: from the sync's point of view, thousands of files were deleted on one side, and the rule says deletions cross. The limit is relative to what was agreed: with 2,873 files in the archive, up to 1,436 deletions one way is a normal (if large) run; 1,437 stops it. For small trees the floor of 20 keeps a handful of deletes from tripping it. And a side whose tree is entirely empty while the archive holds files always stops, whatever the count.

Agreement Only From the Version Still There

After a run moves files, it rebuilds the archive from a second listing of both trees. That is where a subtle bug would hide, and the guard against it was in the first commit, written into Delta's house sign as a gotcha so no later rewrite drops it. Suppose a file was hashed during the run (it was a maybe) and then edited before the run finished. If the new archive recorded the hash together with the new mtime from the second listing, the edit would be written down as already agreed, and it would never cross. So a hash vouches only for the version it read: an entry is refreshed only when the mtime seen in the second listing is the one the hash was taken against. Otherwise the last agreement stands, and the next run sees the edit as a change.

A Quiet Run Costs One Listing

Most runs of a pair that runs every minute have nothing to do. Before the 1110-7 review, even those listed both trees twice: once to plan, once to rebuild the archive. Now a run that changed nothing reads nothing back; the listings it planned from are the trees as it left them. For a pair that runs every minute, that halves the listing cost of every minute that did nothing, which matters when the same Mac also answers other apps' listings.

Code

Stop, clock_skew and big_delete (trimmed excerpt)·python

# delta/pair.py (trimmed excerpt): the stops, all before the first keep or transfer
BIG_DELETE_MIN = 20          # deletions on one side beyond max(this, half the archive) stop a run
BIG_DELETE_SHARE = 0.5


class Stop(Exception):
    """A reason no retry clears by itself; nothing was changed."""

    def __init__(self, cause: str, message: str):
        super().__init__(message)
        self.cause, self.message = cause, message


def clock_skew(runner, peer: str) -> float:
    """The other Mac's clock minus this one's, half the round trip taken out."""
    started = time.time()
    doc = runner.delta.peers.request(peer, "GET", "/v1/health", app="delta-pair").raise_for_status().json()
    ended = time.time()
    if not isinstance(doc, dict) or "time" not in doc:
        raise Stop("clock_unread", f"{peer}'s Delta does not report its clock; update it")
    return float(doc["time"]) - (started + ended) / 2


# inside sync(), after both trees are listed and every path classified:
agreed_n = len(A)
for side, files, deletes in ((here.mac, L, del_there), (there.mac, R, del_here)):
    if agreed_n and not files:
        raise Stop("big_delete", f"the whole tree on {side} appears deleted ({agreed_n} files agreed at the "
                                 "last run); nothing ran")
limit = max(BIG_DELETE_MIN, int(agreed_n * BIG_DELETE_SHARE))
if len(del_there) > limit or len(del_here) > limit:
    raise Stop("big_delete", f"{len(del_there)} deletions toward {there.mac} and {len(del_here)} toward "
                             f"{here.mac} exceed {limit} ({agreed_n} files agreed): nothing ran")

Two preflight checks you can run: clock skew against a local health endpoint, and the big-delete limit·python

# preflight.py: two stops a two-way sync checks before it changes anything
import json
import threading
import time
import urllib.request
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

FAKE_OFFSET = 75.0          # pretend the other Mac's clock runs 75 s ahead
MAX_SKEW_S = 60.0
BIG_DELETE_MIN, BIG_DELETE_SHARE = 20, 0.5


class Health(BaseHTTPRequestHandler):
    def do_GET(self):
        body = json.dumps({"ok": True, "time": time.time() + FAKE_OFFSET}).encode()
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    def log_message(self, *args):
        pass


def clock_skew(url: str) -> float:
    """The peer's clock minus ours, half the round trip taken out."""
    started = time.time()
    with urllib.request.urlopen(url, timeout=5) as resp:
        doc = json.load(resp)
    ended = time.time()
    return float(doc["time"]) - (started + ended) / 2


def big_delete(agreed: int, deletes_there: int, deletes_here: int, files_here: int, files_there: int):
    if agreed and (files_here == 0 or files_there == 0):
        return "a whole tree appears deleted"
    limit = max(BIG_DELETE_MIN, int(agreed * BIG_DELETE_SHARE))
    if deletes_there > limit or deletes_here > limit:
        return f"{max(deletes_there, deletes_here)} deletions exceed {limit}"
    return None


server = ThreadingHTTPServer(("127.0.0.1", 0), Health)
threading.Thread(target=server.serve_forever, daemon=True).start()
skew = clock_skew(f"http://127.0.0.1:{server.server_port}/v1/health")
print(f"skew {skew:+.1f} s ->", "stop: clock_skew" if abs(skew) > MAX_SKEW_S else "ok")
for case in [(2873, 12, 0, 2873, 2861), (2873, 1500, 0, 2873, 1373), (30, 0, 0, 30, 0), (40, 21, 0, 40, 19)]:
    reason = big_delete(*case)
    print(case, "->", f"stop: big_delete ({reason})" if reason else "ok")
server.shutdown()

Exercise

Run the preflight demo, then change FAKE_OFFSET to 45 and to -90 and note which runs stop. Next, pick a folder you sync between two places today (a cloud drive, a backup tool, rsync in a script) and answer for it: what does it do if one side's root is missing, if one side's clock is two minutes off, and if a folder will not open? Write down which of Delta's stops your tool has and which it lacks.

Hint

Many one-way tools treat a missing source as an empty one and, with a delete option, empty the destination. rsync's protection is --max-delete; check whether your script passes it. For the clock, compare date on both machines with a reference such as sntp time.apple.com.