A Plain Tree, and the Links That Never Lead Out

The Home, the Way scp Sees It

Delta's first child is the one every Mac needs: DirectoryShare, a plain tree under one root folder. Its own child, HomeShare, is that tree rooted at the account's home, and its docstring sets the expectation in one line: the home the way an scp user sees it, minus credential stores and the folders macOS guards. Before Delta, moving a file between Macs meant scp or rsync over ssh, and a person typing ~/ on the far side meant the far account's home. Delta keeps that mental model, so delta cp paths read the way people already type them.

The defaults say who the home is for. HomeShare sets readers and writers to the whole fleet (every family Mac but the household's), makes the share writable unless this Mac's host config says writable: false, and keeps the macOS privacy guard on. A second built-in, volumes over /Volumes, exists but is off by default: removable and network volumes are guarded for a launchd python too, so a Mac enables that share only after granting the access, never by default.

Clean the Path Before Anything Touches the Disk

Every address a caller sends is a share-relative path, and every path goes through clean_rel first. It is three rules, all lexical, all decided before a single system call:

Input	Result
`docs/./a.txt`	`docs/a.txt` (empty and `.` parts dropped)
`docs//a.txt`	`docs/a.txt`
`docs/../../etc`	`Forbidden`: `'..' is not a path component Delta follows`
a path containing a NUL byte	`BadRequest`: `a path may not contain NUL`

Note what clean_rel does not do: it does not try to resolve .. against its neighbours and carry on. A path that climbs is refused outright. Normalizing a traversal attempt into something that happens to land inside the tree is how servers end up serving paths nobody meant to allow; refusing it costs nothing, because no legitimate caller needs to climb.

A Link Never Leads Out

A clean path is not yet a safe one. Suppose a folder in the home holds a symbolic link, docs/shortcut, that points at some folder outside the share. The path docs/shortcut/secret.txt has no .. in it; clean_rel passes it untouched. Only the filesystem knows where it really goes.

So every read is judged a second time on the path with every link followed. _resolved takes os.path.realpath of the share root and of the requested path, and refuses with <rel> is a link that leaves the share unless the real path is the root or sits under it. It then checks the real, inner path against the same refusal lists the lexical path faced (the next lesson's subject), so a link that points at a credential folder inside the home is refused like the folder itself. The rule is written down in a docstring that is worth quoting in spirit: the refusals are rules about places, so a link that points at one is refused like the place.

Cheap in the Common Case

Following every link on every read sounds expensive, and in Delta's first build it was. A bundle of 400 small files asks 400 times about the same handful of folders, and each answer cost some thirty lstat calls plus the privacy guard's walk. The fix is a short memory: _folder_ok judges a folder once (it resolves inside the share, away from refused places, and holds no guarded folder) and remembers that verdict for FOLDER_OK_SECONDS, five seconds, in a table capped at 20,000 folders. After that, a plain file in a folder already judged costs one lstat. A link is never given the shortcut: whatever it points at is judged with links followed, every time.

Listings got the same treatment. Entries in a listing carry no Path object at all, because nothing opens a file from a listing (only file() does, and it judges again). Building 17,861 path objects for the site's content tree had been a third of what was left of a listing's time.

Writes Never Go Through a Link

Writable trees add one more rule. Before Delta creates, replaces or moves anything, _landing judges the folder the path will sit in (the nearest one that exists, when the folders above it are still to be made) with links followed. Then _write_check refuses the leaf itself if it is a link: Conflict, <rel> is a symbolic link; Delta does not write through links. A write that followed a link could land anywhere the link points; refusing it keeps every byte Delta writes inside the tree it was asked about. remove deletes only empty folders (a sync deletes the children first), and whatever keep sets aside lands create-only: a kept version is a hard link to the file itself, a copy is written to a temporary file and then hard-linked into place, and a link is recreated with os.symlink. Those three fail rather than replace a file someone else put there. The one exception is a link that keep moves (a delete that keeps it): it is renamed into a name that was free when checked, and os.rename would replace a file that appeared in the instant between the check and the rename.

Links still cross between Macs; they cross as links, listed with their stored target and recreated on the other side, never followed and never written through. Track 6 shows how a transfer decides which targets are allowed.

Code

The lexical clean and the link-followed check (trimmed excerpt)·python

# delta/shares.py — clean_rel and DirectoryShare._resolved (trimmed excerpt)

def clean_rel(rel):
    """A share-relative path, normalized; '..' and NUL are refused outright."""
    text = rel or ""
    if "\x00" in text:
        raise BadRequest("a path may not contain NUL")
    parts = [p for p in text.split("/") if p not in ("", ".")]
    if any(p == ".." for p in parts):
        raise Forbidden("'..' is not a path component Delta follows")
    return "/".join(parts)


class DirectoryShare(DataShare):
    def _resolved(self, path, rel):
        """path with every link followed, proven to stay inside the share and
        outside what the share refuses. The refusals are rules about places, so a
        link that points at one is refused like the place itself."""
        root = os.path.realpath(self._root)
        real = os.path.realpath(path)
        if real != root and not real.startswith(root.rstrip("/") + "/"):
            raise Forbidden(f"{self.id}/{rel} is a link that leaves the share")
        inner = real[len(root):].strip("/")
        reason = self._denied(inner) if inner else None
        if reason:
            raise Forbidden(f"{self.id}/{rel}: refused, it is a link to a place where {reason}")
        if self.tcc_guard and real != path:
            self._guard(real, rel)
        return real

Runnable: a clean path that still escapes, caught only by following the link·python

# Two checks a plain-tree share needs: a lexical clean, then the path with every link followed.
import os
import tempfile


class Refused(Exception):
    pass


def clean_rel(rel):
    """Normalize a share-relative path; NUL and '..' are refused outright."""
    text = rel or ""
    if "\x00" in text:
        raise Refused("a path may not contain NUL")
    parts = [p for p in text.split("/") if p not in ("", ".")]
    if any(p == ".." for p in parts):
        raise Refused("'..' is not a path component this share follows")
    return "/".join(parts)


def resolved(root, rel):
    """The real path behind rel, proven to stay inside root."""
    real_root = os.path.realpath(root)
    real = os.path.realpath(os.path.join(root, clean_rel(rel)))
    if real != real_root and not real.startswith(real_root.rstrip("/") + "/"):
        raise Refused(f"{rel} is a link that leaves the share")
    return os.path.relpath(real, real_root)


if __name__ == "__main__":
    with tempfile.TemporaryDirectory() as outside, tempfile.TemporaryDirectory() as root:
        with open(os.path.join(outside, "secret.txt"), "w") as f:
            f.write("not yours")
        os.makedirs(os.path.join(root, "docs"))
        with open(os.path.join(root, "docs", "a.txt"), "w") as f:
            f.write("yours")
        os.symlink(outside, os.path.join(root, "docs", "shortcut"))
        for rel in ("docs/./a.txt", "docs//a.txt", "docs/../../etc", "docs/shortcut/secret.txt"):
            try:
                print(f"{rel!r:28} clean={clean_rel(rel)!r:26} serves {resolved(root, rel)}")
            except Refused as exc:
                print(f"{rel!r:28} refused: {exc}")

Exercise

Run the runnable block and confirm the four verdicts. Then add a fifth case: a symbolic link inside root that points at another folder that is also inside root (for example docs/latest pointing at docs). Predict whether docs/latest/a.txt is served before you run it, and explain why the same rule that refused shortcut lets this one through.

Hint

Create it with os.symlink("../docs", os.path.join(root, "docs", "latest")) or an absolute target inside root. The realpath lands under the real root, so it is served. The rule is about where the path really goes, not about whether a link was involved.