Four Wires, One Order

Four Roads, One Order

Once a peer is known to be in the same house, there may still be more than one way to reach it. Two Macs on one desk can share a Thunderbolt cable, the house's 10GbE switch and the Wi-Fi all at once, and each of those addresses will pass the on-link test. Delta does not pick among them by luck. Every candidate carries the kind of link it was found on, and the list is sorted by a fixed preference before a single probe goes out:

Thunderbolt bridge: a cable between exactly two Macs.
Ethernet: the house's wired network.
Wi-Fi: the house's wireless network.
The tailnet: everything else, and the fallback for all of the above.

The kind comes from the hardware port's own name. networksetup -listallhardwareports calls the bridge "Thunderbolt Bridge"; a title with "Wi-Fi" in it is Wi-Fi; a title with "Ethernet" in it, or any other port whose name begins with "Thunderbolt", counts as Ethernet. A candidate is a thunderbolt route only when both ends say so: this Mac's interface is the bridge and the peer advertised that address as its bridge. Anything else on-link becomes a lan route. The candidates are then probed in order, and the first one to answer /v1/health is the route. If every one fails, the problems are joined into the reason and the tailnet takes over.

Why This Order: The Numbers

The preference is not taste. It is a table, measured on 2026-11-09 from the workbench to the service host with a 1 GiB transfer:

Path	Throughput
tailnet (running directly over the LAN)	0.32–0.36 Gbit/s
10GbE LAN, plain HTTP	9.0–9.2 Gbit/s
Thunderbolt bridge, plain HTTP	28.3 Gbit/s
TLS in Apple's python3 (LibreSSL 2.8.3), one stream / four	1.9 / 3.6 Gbit/s

Read the first row carefully. The tailnet number is with both Macs in the same room, the tunnel riding straight over the LAN. That is the cost of the tunnel itself, not of the internet. Against the middle of its range, the 10GbE wire is about 27 times faster and the Thunderbolt cable about 83 times; Wi-Fi was not part of this measurement, so its place in the order is a design choice, not a row. Read the last row too: on the 10GbE LAN, one TLS stream from Apple's python3 ran at 1.9 Gbit/s against plain HTTP's 9.0 to 9.2, nearly five times slower, and four streams at 3.6 still left more than half the wire unused. Set the same stream against the Thunderbolt cable's 28.3 and it is fifteen times slower; those two ratios are where the architecture doc's "a factor of about 5 to 15" comes from. Together the rows decide not only the order but how each road is secured.

Three Roads, Three Kinds of Protection

Thunderbolt carries plain HTTP with a ticket. A Thunderbolt bridge is a cable whose only other end is the peer. Nobody else is on it to listen or to inject, so encryption would buy nothing and cost most of 28 Gbit/s. The caller still proves who it is with a ticket the peer minted over the tailnet (track 4).
Shared Ethernet and Wi-Fi carry TLS plus a ticket. Those wires are shared with every other device in the house: televisions, guests' phones, a printer running firmware from 2019. Here the TLS is worth its price, and it is pinned to the certificate fingerprint the peer published over the tailnet.
The tailnet carries plain HTTP inside WireGuard. The tunnel already encrypts every packet and binds every source address to a node key. Stacking a second TLS handshake inside it would pay the encryption tax twice for nothing. Dad's call on 2026-11-09: no second handshake on the tailnet.

The Serving Side Agrees Without Being Told

A route is a decision both ends must make the same way, and the gateway on port 5201 makes it independently, from the address each connection arrived on. If both the caller's and this Mac's addresses are tailnet addresses (the 100.64.0.0/10 range), the connection is a tailnet connection. Otherwise the gateway looks up which physical link owns its own local address: the bridge means plain HTTP; any other physical link means the very first byte must be 0x16, the start of a TLS handshake, or the connection is refused with a log line. A caller from a non-private address, or on an address that belongs to no link the gateway knows, is closed without an answer. The asking side cannot talk a LAN route into plain HTTP, because the serving side never asks it what it would like.

Where the TLS Tax Comes From

The LAN's ceiling is Apple's python3 itself. It links LibreSSL 2.8.3, which gives AES-GCM no hardware help. The architecture doc puts the two ciphers at about 400 MB/s per stream for ChaCha20 and 115 to 150 MB/s for AES-GCM there, which is why Delta's TLS context offers ChaCha20 first. Those are the cipher figures; a whole transfer also hashes and writes every byte, and its measured rate is lower, the 1.9 Gbit/s (about 240 MB/s) of one stream in the table. So on the shared LAN Delta spreads a big file across six parallel streams. After the 1110-7 review, six streams moved 4 GiB over TLS at 6.7 to 8.7 Gbit/s, just under the 10GbE link (the architecture doc's before-and-after table; track 6 shows its Thunderbolt column). On Thunderbolt Delta uses eight streams, and on the tailnet three (track 6 takes the transfers apart). The worker counts are the route table turned into concurrency.

Code

The preference order and the gateway's own classification (trimmed excerpt)·python

# delta/routes.py and delta/server.py: the order on the asking side, the class on the
# serving side (trimmed excerpt)

LINK_ORDER = {"thunderbolt": 0, "ethernet": 1, "wifi": 2}

# asking side, inside RouteBook._select, for each on-link candidate:
local_link = links.get(nic, {}).get("link", "ethernet")
kind = "thunderbolt" if local_link == "thunderbolt" and link == "thunderbolt" else "lan"
candidates.append(Route(peer, kind, endpoint["address"], gateway_port,
                        tls=(kind == "lan"), fingerprint=ident.get("tls_sha256", ""),
                        interface=nic, link=local_link))
candidates.sort(key=lambda r: (LINK_ORDER.get(r.link, 9), r.kind != "thunderbolt"))
# ...then probe each in that order; the first that answers /v1/health is the route.


# serving side, the gateway classifying a connection by the address it arrived on:
def admit(self, sock, peer_ip, local_ip):
    if identity.is_tailnet(peer_ip) and identity.is_tailnet(local_ip):
        return Conn("tailnet", peer_ip, local_ip, sock)          # WireGuard names the caller
    link = link_class_for_local_address(local_ip, self.links())
    if link is None or not ipaddress.ip_address(peer_ip).is_private:
        log.warning("gateway refused %s -> %s: not a tailnet, Thunderbolt or home LAN path",
                    peer_ip, local_ip)
        return None                                              # closed unanswered
    if link == "thunderbolt":
        return Conn("thunderbolt", peer_ip, local_ip, sock)      # a cable: plain HTTP + ticket
    return self._tls_wrap(sock, Conn("lan", peer_ip, local_ip, sock), peer_ip)  # TLS or nothing

tlscost.py: measure the TLS tax on your own python·python

# tlscost.py: what TLS costs one stream on this Mac's python, over loopback.
# First, in the same folder:
#   openssl req -x509 -newkey rsa:2048 -nodes -keyout key.pem -out cert.pem -days 1 -subj "/CN=localhost"
# Then run it with the python you care about, e.g. /usr/bin/python3 tlscost.py
import socket
import ssl
import sys
import threading
import time

TOTAL = 256 * 1024 * 1024        # bytes to push through each run
BLOCK = 1024 * 1024


def sink(server_sock, wrap):
    conn, _ = server_sock.accept()
    if wrap:
        conn = wrap(conn)
    while conn.recv(BLOCK):
        pass
    conn.close()


def run(cipher=None):
    server = socket.socket()
    server.bind(("127.0.0.1", 0))
    server.listen(1)
    wrap = client_wrap = None
    if cipher:
        sctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
        sctx.load_cert_chain("cert.pem", "key.pem")
        sctx.maximum_version = ssl.TLSVersion.TLSv1_2   # so the cipher below is the one used
        sctx.set_ciphers(cipher)
        wrap = lambda s: sctx.wrap_socket(s, server_side=True)
        cctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
        cctx.check_hostname = False
        cctx.verify_mode = ssl.CERT_NONE                # a benchmark, not an identity check
        cctx.set_ciphers(cipher)
        client_wrap = lambda s: cctx.wrap_socket(s)
    t = threading.Thread(target=sink, args=(server, wrap))
    t.start()
    client = socket.create_connection(server.getsockname())
    if client_wrap:
        client = client_wrap(client)
    payload = b"\0" * BLOCK
    started = time.monotonic()
    for _ in range(TOTAL // BLOCK):
        client.sendall(payload)
    client.close()
    t.join()
    seconds = time.monotonic() - started
    print(f"{cipher or 'plain TCP':32s} {TOTAL * 8 / seconds / 1e9:6.2f} Gbit/s")


print(ssl.OPENSSL_VERSION, "/ Python", sys.version.split()[0])
run()
run("ECDHE-RSA-CHACHA20-POLY1305")
run("ECDHE-RSA-AES128-GCM-SHA256")

Exercise

Generate the throwaway certificate and run tlscost.py twice: once with Apple's /usr/bin/python3 and once with any other Python you have (Homebrew, conda, pyenv). Record the library version line and the three numbers for each. Then, using your fastest TLS figure and your plain-TCP figure, decide for your own house which links you would encrypt and which you would protect with identity alone, and write one sentence of justification per link.

Hint

The first line prints ssl.OPENSSL_VERSION: Apple's python3 reports LibreSSL, most other builds report OpenSSL 3, which uses the CPU's AES instructions and often flips the ChaCha20 versus AES-GCM ranking. If a cipher name is refused, your library spells it differently: list what it offers with ssl.create_default_context().get_ciphers().