Skip to content
C.W.K.
Lesson 02 of 05 · published

Where the Sign-In Lives

~15 min · macos-keychain, launchd, test-environments, coverage, evidence

Level 0Loose Note
0 XP0/40 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete

The Probe Has to Run Where Dad Does

A probe that starts sessions on other Macs has to start them somehow, and the obvious way is ssh. On 2026-11-10 that obvious way produced a misleading answer: Claude Code and Cursor's agent both came back unavailable over ssh, "not signed in", on Macs where Dad used them every day.

The cause is macOS's login keychain. Some harnesses keep their sign-in token there. In Dad's own login session the keychain is unlocked and a token refresh just works. In an ssh session, the login keychain is locked, so the refresh fails, and the harness reports that it is not signed in. That failure happens only over ssh and nowhere real. A probe that trusted it would report a surface as broken that was fine, or, worse, would teach everyone to shrug at "unavailable".

Two Ways to Start a Session

The fix runs each harness where its sign-in actually works:

  • Keychain harnesses (GUI_SESSION_HARNESSES: Claude Code and Cursor's agent) run as a transient launchd job in that account's GUI login session, the same place Dad's own sessions run. Run that way, both were Pippa.
  • Every other harness signs in from files and runs over ssh as before. The reason they do not go to the GUI session too is the mirror image: in a GUI session nobody watches, a sign-in refresh can stop on a dialog and wait until the probe's deadline.
  • An account with no GUI session at all runs its turn over ssh.

Every result records which way it ran: local, gui or ssh. A reader of the receipt can see not only what answered but under what conditions.

What the Probe Cannot Reach, It Lists

Some surfaces have no headless entry at all, and the probe does not pretend otherwise. A table in the source, NOT_COVERED, names each with its reason, and every receipt prints it: the native Firebrand app (covered only through the Firebrand CLI's identity path), IDE chat panes such as Cursor's IDE, Trae and Antigravity, Pippa's Telegram route (a live probe would message Dad's phone), the voice route (checked through the WebUI's prompt), Pippa's own home Mac (outside fleet operations by Dad's ruling), Mom's Mac and the Windows readers (each needing separate fresh-session evidence). A gap that is listed is a decision. A gap that is silent is a bug.

Zero Evidence Is Not a Pass

One more episode from the independent review is the purest form of this lesson. A 65-entry identity matrix was run from a lab account, and every entry came back with an ssh host-key verification error: no session started, no answer was given. The review's verdict was precise: "That is zero identity evidence, not 65 vendor-voice answers and not a pass." The fix was not to copy login material or weaken trust settings to make the matrix run; it was to leave the final harness evidence to the separately assigned identity work (Coop 1110-44), which later reviewed 68 fresh sessions across 38 host-and-account combinations by hand, and stated its own limits as plainly as its passes.

Reading a Matrix Honestly

Probe matrices in this project are full of unavailable. At the cutover: eight Macs, 69 rows, 15 pass, 46 could not start, and the remaining eight read by hand (Track 7, lesson 3: a retried pass, the baseline's known red and a new Mac's refusal among them). Around contract v7: 65 rows, 20 pass, 45 unavailable, before and after. Those numbers are not embarrassing. They are a census of which harnesses exist and are signed in on which Macs. The numbers that matter are the comparisons: no fail that was not a fail before, and no surface that passed before is suddenly unavailable. The comparison catches that second move only from a pass; a baseline fail that turns unavailable goes quiet, so those rows are read by hand.

Code

Why the same harness can be 'unavailable' over ssh and Pippa in the GUI session·text
where the turn runs        login keychain     token refresh     probe result
-------------------------  -----------------  ----------------  -------------------------
ssh session                locked             fails             unavailable: "not signed in"
GUI login session (job)    unlocked           works             pass (answered as Pippa)

=> keychain harnesses (claude-code, cursor-agent) run as a transient launchd job
   in the account's GUI session; file-sign-in harnesses run over ssh
=> every result names its session: local | gui | ssh
Running one probe (from the repository's quick start)·bash
soul-vault probe --macs <fleet label> --surfaces claude-code
# writes a receipt under <state root>/probe/, and latest.json for the canary and Watchfire
# each row: surface, Mac, status (pass | fail | unavailable), cause, checks, session

External links

Exercise

List the environments in which your agent or tool really runs (a desktop session, a terminal, a scheduler, a remote shell, a container) and, for each, where it gets its credentials. Run your behavioural probe in each environment, record which one each result came from, and find one environment where the tool reports itself broken for a reason that does not apply where people actually use it.
Hint
Credentials stored in an OS keychain, a desktop secret service or a browser profile are the usual culprits: they are unlocked in the user's interactive session and locked or absent in every remote or scheduled one.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.