Skip to content
C.W.K.
Lesson 10 of 10 · published

Say Why the Link Is Down, Then Go Quiet

~15 min · mobile, network, honest-failure, tailscale, measurement

Level 0Curious
0 XP0/102 lessons0/18 achievements
0/100 XP to next level100 XP to go0% complete

One honest sentence, wrong twice

On 2026-10-13 Dad was abroad and kept losing the engine, and the cause was usually the device in his hand: Tailscale had dropped on the phone. The app said "Connection lost" or "Reconnecting" and never said that. So the kit gained one sentence, "Check that Tailscale is on and connected on this device.", and the web, the phone and the watch began to carry it on network failures only, never on an engine that had answered. The same evening it learned its first exception: a request to the Mac itself, on loopback, is never a Tailscale problem.

The sentence was honest, and in the next ten days it misled twice, both times by being said where it was not true. On 2026-10-18 in Málaga, four photos going up over LTE held the one kept-alive connection; the small reads behind them timed out, and the hint appeared over a turn that was running fine (lesson 2's note). On 2026-10-23 at 02:32 KST, still the evening of the 22nd in Málaga, Dad was on the MacBook debugging the iPhone's real Tailscale trouble, and the WebUI showed the same red advice under "Connection lost — waiting for server…". The Pippa machine's log showed his MacBook's requests flowing until a restart at 02:31:56 and again within a minute. The engine had been restarting; the tailnet was fine. In a browser, a restart's cut reads as the same TypeError a dropped tailnet throws.

멍 때리거나 엉터리 안내하는 UI가 죄악인 거야. — 아빠

("A UI that stares blankly or gives bogus directions is a sin.") Both halves of that sentence are this lesson. The bogus direction is advice the failure did not earn. The blank stare is a screen that shows nothing changing while something is, or nothing at all while nothing can.

A check decides, never the error

Since 2026-10-23 the cause of a lost link is decided by one quick read of /api/health, through the kit's LostLink, and never guessed from the error. A status that is not a gateway's means the engine itself answered. A 502, 503 or 504 means something answered for the engine: the Tailscale Serve twin in front of it is a reverse proxy, and it answers that way while the engine is down, which proves that this device and the tailnet work. That is a restart, and launchd brings it back, so the line says "Pippa is restarting, back in a few seconds." with the seconds ticking; past 90 seconds it stops promising and says the engine is still not answering while the machine does, so this is not the network. Nothing answering on a device with no network is offline. Nothing answering on a device that is online is the only case that names Tailscale.

The web half is kitLostLink.ts, and a kit test holds its sentences equal to the Swift ones. On the phone the same read changed one line: a transcript read that failed during a restart used to say the conversation was "slow to arrive over this link", because the old check counted the twin's 502 as an answer. It now says the engine is restarting.

Principle: A failed request has more than one possible author: the device, the tunnel, a proxy answering for a stopped server, the server. Ask once, cheaply, which one it was before saying anything, and give advice only for the cause the check proved. When the cause is the tunnel, stop adding to its load, and say on screen that you stopped.

Then go quiet on the wire

The trouble Dad was debugging that evening was real. On 2026-10-22, on steady hotel Wi-Fi, his phone lost the Pippa machine: "A server with the specified hostname could not be found", the Tailscale switch greyed out, and only a reboot brought it back. From the Pippa machine the phone looked Online in the netmap, with no WireGuard handshake and tailscale ping timing out. Two open Tailscale iOS issues on 1.104 carry that signature. In #21773, packets to a peer whose handshake has not completed fill the network extension's 64-buffer pool and stall every peer and MagicDNS until the handshake gives up, which it never does while packets keep arriving; a gap of about 90 to 100 seconds lets it go. In #21832, the DERP receive loop stops, and a restart recovers.

The app never gave the tunnel that gap. Its progress check ran every 3 s with no backoff, beside the stage poll, the private poll, drop retries, timed refreshes and location reports. So the background checks now rest together. LinkRest counts link failures only: a name that does not resolve, nothing connecting, a dropped connection, no reply in time. An engine that answered with an error does not count, because the tunnel carried it, and it does not clear the count either; only a good answer does. Two link failures with no good answer between them stop the background checks for 120 s, longer than the handshake needs to give up; timeout, HTTP 500, timeout counts as two. A send, a pull to refresh, the app coming to the front or the network returning ends the rest at once, and the count stays, so one more failure rests again; a good answer clears it and drains what the rest held back. One check does not rest: when a step stays quiet past its limit, the check described below still reads /api/health, at most every 15 s, and its answer never touches the count. The wire goes quiet and the screen does not: the progress strip says what failed and how many seconds are left, and that a send or a pull tries now.

Two decisions in that work are worth copying. The rest lives in the phone app, not in the kit's pool, because the kit's pool serves every family app and this was cwkPippa's own polling; the kit can inherit it once a recorded outage confirms the mechanism. And the record was built the same night. scripts/phone-link-recorder.py installs two launchd agents on the Pippa machine. One writes a JSON line every 30 s about how Dad's phone reaches it: the phone peer's state and handshake age, a disco ping and a TSMP ping, both DERP homes, a timed request to Tailscale's control plane, and the phone's requests in the access log. The other taps tailscaled's own log and marks the phone's lines. It reads and never writes to tailscaled, and it runs on /usr/bin/python3, outside the Local Network gate. An outage like that reads three ways, our move or our app, the phone's Tailscale, or a Tailscale-wide glitch, and until then nothing recorded the minutes between. Its README row says that its own pings are a workaround one of those issues reports, so a quiet stretch on the record is not misread as health.

Name what moves; "stalled" is a diagnosis

The blank stare had its own walk. On 2026-10-18, on the Alcazaba in Málaga over LTE, two spoken turns with pasted photos read "Uploading your recording · stalled, nothing for 70s" while photos, not a recording, crossed the link. Dad's rule from that walk: no step sits as one frozen word, and "stalled" is a diagnosis, not a timer. Now the transport reports a request body's bytes as they are sent, and the line names the leg with more left to send, with its bytes and rate: "Uploading photo 2 of 4 · 1.1 of 3.6 MB · 40 KB/s". Thinking and writing show the characters that have arrived; a step only the server can move shows when the Pippa machine last answered the stage board (frontend track, lesson 3). Past a step's limit the phone checks its own link, then the Pippa machine, again every 15 s while the step stays quiet, and the line says what it found: no bytes out while the machine answers means the cellular link is not carrying them; a silent machine is said to be silent; no network is said as no network. The watch shows the phone's own line, at most once a second while its app is in front, and ages it when it stops coming.

Two pools and a fresh stream

Lesson 8 of this track ended with one kept-alive pool for every call and the turn stream. Two failures in Málaga took that apart. On 2026-10-19, on hotel Wi-Fi with AirPods, a reply's spoken takes and the turn stream filled the pool's six connections; the 3 s progress check queued behind them, hit its 5 s timeout, and one miss put "Reconnecting" on screen against a Pippa machine answering every call. Spoken takes, uploads, downloads and video now ride a bulk pool of their own, and "Reconnecting" became a verdict rather than a reflex: two failed checks over at least 6 s with nothing heard from the machine for 15 s, with the failing check named. Then, on build 157 at the same hotel, the remaining pool's kept-alive connection went stale six minutes after launch, and a dictated turn sat 49 s on the phone before its POST while the screen said only "not sent for 37s". The turn's stream now opens a fresh connection every turn: one handshake, well under a second from Málaga, paid so that a turn never waits on a dead connection. The kit's pool now heals after two timeouts in 20 s, and a turn still on the phone says how many other calls are open beside it.

The voice side of the same fixes, the reading back and the microphone, belongs to the Voice Mode Quest. What belongs here is the last thing the code shows. As of 2026-10-27 the comment above the first pool in PippaClient.swift still says every call and every turn stream share it, and so does the house sign's rule about the phone's network; the initializer a few lines below says otherwise, and the initializer is what runs. A rule outlives its reversal in every copy nobody went back for.

Code

verbatim, cwkSiblingKit CwkKit/Sources/CwkKitCore/LostLink.swift, as of 2026-10-27·swift
  /// The cause one health read proves. `status` is set when anything answered;
  /// `online` is false when the device has no network path at all.
  public static func classify(status: Int?, error: Error?, online: Bool, loopback: Bool = false) -> LostLink {
    if let status { return gatewayStatuses.contains(status) ? .restarting : .answering }
    guard let error, ConnectionHint.isNetworkOrLoopbackDown(error) else { return .unknown }
    if !online { return .offline }
    // A refused connection on this machine is the engine itself being away.
    if loopback { return .restarting }
    return .unreachable
  }
verbatim, cwkPippa clients/apple/PippaMobile/Sources/LinkRest.swift, as of 2026-10-27·swift
    /// A background check failed. Returns true when this failure began a rest.
    @discardableResult
    mutating func failed(_ error: Error, at now: Date) -> Bool {
        guard Self.isLinkFailure(error) else { return false }
        failures += 1
        lastProblem = (error as? LocalizedError)?.errorDescription ?? error.localizedDescription
        guard failures >= Self.restAfterFailures, !resting(at: now) else { return false }
        until = now.addingTimeInterval(Self.restSeconds)
        return true
    }
The phone's connections since 2026-10-20 (sketch)·text
control pool (kept alive)   reads, polls, progress, the grant, take tickets
bulk pool (kept alive)      spoken takes, uploads, downloads, video
fresh connection            the turn's stream, one per turn, one handshake

a lost link -> one GET /api/health decides
  any other status      answering    "Pippa answers again. Catching up…"
  502 / 503 / 504       restarting   the twin answered FOR a stopped engine
                                     (no advice; past 90 s, no promise)
  nothing, no network   offline      "This device is offline."
  nothing, online       unreachable  the only line with the Tailscale advice

two link failures, no good answer between -> the background checks rest 120 s
  woken by: a send, a pull to refresh, the app in front, the network back
  cleared by: a good answer (then the held drops go);
              an engine error neither counts nor clears
  not resting: the health read for a step quiet past its limit
  the strip says what failed and the seconds left

External links

Exercise

Find the line your client shows when a request fails. List every cause that can produce it: the device offline, the tunnel, a proxy answering for a stopped server, the server itself, an upload holding the only connection. For each, write the one cheap check that tells it apart from the others and the sentence the user should see. Then watch what your client's background work sends during the failure, and count it per minute.
Hint
The test of a hint is whether it could be shown while the cause was something else. And a client that retries every three seconds into a wedged tunnel is part of the outage, not a witness to it.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.