Lesson 05 of 05 · published

Two Listeners, One Mac

~14 min · architecture, loopback, gateway, launchd, ports

Level 0Loose File

0 XP0/43 lessons0/12 achievements

0/100 XP to next level100 XP to go0% complete

Two Doors, Two Audiences

One Delta process runs on each family Mac, and it listens on exactly two sockets. They are not two copies of one server. They face two different audiences, and almost every rule in Delta follows from keeping them apart.

	engine	gateway
Bound to	`127.0.0.1:5200`	`0.0.0.0:5201`
Who calls it	The apps on this Mac	The Deltas on the other Macs
Proof of who is asking	The local token, `X-Delta-Local-Token`	Depends on the route the connection arrived on
What it does	Reads other Macs' shares through the cache, starts transfer jobs, reports health	Serves this Mac's shares, mints tickets, receives pushed files

An app that wants a file from the workbench does not talk to the workbench. It talks to its own Mac's engine, on loopback, naming itself in X-Delta-App. The engine checks its cache, picks a route, and asks the workbench's gateway. That is how the last lesson's rule holds: the app's own Python never dials the LAN, and the engine's address is one no other machine can even reach.

The Engine: Loopback and a Token

The engine refuses any connection that does not come from a loopback address, before reading a byte of it. Then every route except health and version needs the local token: 32 bytes of randomness in a file called local.token in Delta's state folder, mode 0600, so only the account Delta runs as can read it. Loopback alone is not identity. Any account logged in to the Mac can open a socket to 127.0.0.1; only the owning account can read the file. The comparison uses hmac.compare_digest, which takes the same time whether the first byte or the last one differs.

The Gateway: Classified by Where It Arrived

The gateway binds every interface, so it has to decide who is calling from facts it can verify at the socket. It looks at two addresses, the caller's and the local one the connection arrived on, and sorts each connection into exactly one kind:

loopback: a process on this Mac. It is this Mac only if it shows the local token too. (Before that rule, any account on the Mac read the home share as "this Mac".)
tailnet: both addresses inside the tailnet's range. The caller is whichever node Tailscale's roster names for that address. Plain HTTP, because WireGuard already encrypts and authenticates the tunnel.
thunderbolt: the local address belongs to a Thunderbolt bridge, a cable between two Macs. Plain HTTP plus a ticket.
lan: the local address belongs to another physical interface (Ethernet, Wi-Fi) and the caller's address is private. TLS with a pinned certificate, plus a ticket.
anything else: closed unanswered. No error page, no banner, nothing to fingerprint.

Which physical interface is which comes from the Mac itself: networksetup -listallhardwareports names each device's hardware port (Thunderbolt Bridge, Wi-Fi, Ethernet) and ifconfig gives its addresses. Track 3 follows that information outward into route choice, and track 4 takes the proofs apart.

No Address, Port or Machine Name in Code

You have just read two port numbers. You will not find either in Delta's source as a literal a request uses. Ports come from Firelink's port ledger, where Delta holds two @each claims (one engine and one gateway port, the same on every Mac). Delta's config asks the vendored registry module for them, and if that snapshot predates the claim, it reads the live ledger file rather than guessing a number. If neither knows, it refuses to start with a named error. This Mac's identity comes from its fleet label file, its peers from the registry, and LAN addresses from each peer's own Delta at run time. That is invariant 8, and it is why moving a service between Macs never means editing Delta.

Kept Alive by launchd

The process is a LaunchAgent, com.cwk.delta, with RunAtLoad and KeepAlive: it starts at login and launchd restarts it whenever it exits, waiting at least 10 seconds between starts (ThrottleInterval). Its state lives in one folder, ~/cwk-dbs/delta-db/: the SQLite file, the cache, partial transfers, TLS material, the local token and logs. Connections are HTTP/1.1 keep-alive and idle out after 75 seconds. The serve loop wakes once a second for its signal check and, every six hours, sweeps abandoned partial transfers. Nothing else runs on a timer. Delta has no scheduler: it moves data only when a person or an app asks.

Code

How the gateway sorts a connection (trimmed excerpt)·python

# delta/server.py, _Server.admit (trimmed excerpt; test hooks removed)
def admit(self, sock, client_address):
    peer_ip = client_address[0]
    local_ip = sock.getsockname()[0]
    if self.role == 'engine':
        if not ipaddress.ip_address(peer_ip).is_loopback:
            log.warning('engine refused a non-loopback caller %s', peer_ip)
            return None
        return Conn('engine', peer_ip, local_ip, sock, self.delta.label)
    if ipaddress.ip_address(peer_ip).is_loopback:
        # A process on this Mac: this Mac's label, once it shows the local token.
        return Conn('loopback', peer_ip, local_ip, sock)
    if identity.is_tailnet(peer_ip) and identity.is_tailnet(local_ip):
        return Conn('tailnet', peer_ip, local_ip, sock)
    link = link_class_for_local_address(local_ip, self.links())
    if link is None or not ipaddress.ip_address(peer_ip).is_private:
        log.warning('gateway refused %s -> %s: not a tailnet, Thunderbolt or home LAN path', peer_ip, local_ip)
        return None
    if link == 'thunderbolt':
        return Conn('thunderbolt', peer_ip, local_ip, sock)
    return self._tls_wrap(sock, Conn('lan', peer_ip, local_ip, sock), peer_ip)

Runnable: two listeners in one process, a token on loopback·python

# Two listeners in one process, the way Delta splits its doors.
# engine: loopback only, every route but health needs the local token.
# gateway: any interface, each connection classified by the address it arrived on
# (simplified: Delta also checks the local address and which physical link it is on).
import hmac
import http.server
import ipaddress
import json
import secrets
import socketserver
import threading
import urllib.error
import urllib.request

TOKEN = secrets.token_hex(16)   # Delta keeps its own in a 0600 file


class Server(socketserver.ThreadingMixIn, http.server.HTTPServer):
    daemon_threads = True

    def __init__(self, address, role):
        self.role = role
        super().__init__(address, Handler)

    def classify(self, local_ip, peer_ip):
        peer = ipaddress.ip_address(peer_ip)
        if self.role == 'engine':
            return 'engine' if peer.is_loopback else None
        if peer.is_loopback:
            return 'loopback'
        if peer in ipaddress.ip_network('100.64.0.0/10'):
            return 'tailnet'
        return 'lan' if peer.is_private else None


class Handler(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        kind = self.server.classify(self.connection.getsockname()[0], self.client_address[0])
        if kind is None:
            self.close_connection = True   # closed unanswered
            return
        if kind in ('engine', 'loopback') and self.path != '/v1/health':
            given = self.headers.get('X-Delta-Local-Token', '')
            if not hmac.compare_digest(given.encode(), TOKEN.encode()):
                return self.reply(401, {'error': 'unauthorized', 'detail': 'local token required'})
        self.reply(200, {'role': self.server.role, 'kind': kind, 'path': self.path})

    def reply(self, status, doc):
        body = json.dumps(doc).encode()
        self.send_response(status)
        self.send_header('Content-Type', 'application/json')
        self.send_header('Content-Length', str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    def log_message(self, *args):
        pass


engine = Server(('127.0.0.1', 0), 'engine')
gateway = Server(('0.0.0.0', 0), 'gateway')
for s in (engine, gateway):
    threading.Thread(target=s.serve_forever, daemon=True).start()


def get(port, path, token=None):
    req = urllib.request.Request(f'http://127.0.0.1:{port}{path}')
    if token:
        req.add_header('X-Delta-Local-Token', token)
    try:
        with urllib.request.urlopen(req, timeout=3) as r:
            return r.status, json.loads(r.read())
    except urllib.error.HTTPError as e:
        return e.code, json.loads(e.read())


print(get(engine.server_address[1], '/v1/health'))
print(get(engine.server_address[1], '/v1/shares'))
print(get(engine.server_address[1], '/v1/shares', TOKEN))
print(get(gateway.server_address[1], '/v1/shares', TOKEN))

External links

Exercise

Run the two-listener example, then extend it: write the token to a file with mode 0600 and read it back in the client, add a /v1/version route that needs no token, and make a connection the gateway closes unanswered leave a warning line naming the peer address and the local address it arrived on, the way Delta's admit does. Finally, from a second Mac on your LAN, request the gateway port and confirm which kind the connection is classified as.

Hint

Use os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) so the file is never world-readable, even for a moment. To reach the gateway from another Mac, print gateway.server_address and use this Mac's LAN address with that port; a Conda python on the second Mac may meet the local network gate from lesson 3.