The Loop, and Who Owns Each State

"Okay. Let's go to step 2." — Dad, starting hands-free, 2026-09-25

The Loop on Paper

On paper, a hands-free talk is a circle: idle, listening, sending, thinking, speaking, and back to listening. The design drew exactly that. The implementation taught something the drawing hides: those states don't all belong to the same component. Listening belongs to the microphone loop. Thinking belongs to the chat stream. Speaking belongs to the audio player. A loop that tries to own all of them ends up guessing about the two it doesn't control.

The Listener's Own Phases

So the hands-free loop, on the web and on the phone alike, owns only its own phases: off, connecting (asking the brain for a listening session and opening the socket), listening (streaming 16 kHz audio and showing partial words), waiting (an utterance has ended; it is being kept, sent, and answered), paused and stopped. Thinking and speaking it simply observes. It starts listening again only when two independent facts are both true: the reply's stream has ended, and the player reports that nothing is playing or queued. That second fact needed its own fix on the web, where the player hadn't been releasing finished audio elements, so "is anything playing?" could still answer yes after the last word had played.

The Microphone Stays Open; the Socket Doesn't Always Listen

The browser opens the microphone once per talk, with echo cancellation on, and keeps it. Audio chunks go to the transcriber only while the loop is listening; while the soul thinks or speaks they go nowhere except to the barge-in detector of Track 7. Before barge-in existed, the rule was simpler still: listening ended at the commit and reopened after the reading, so the soul could never hear herself.

A Quiet Room Is Not a Talk

An open realtime socket bills for the audio it hears, silence included. After 60 seconds with nothing heard, the loop stops itself and says so on the face screen. One tap listens again.

The Phone Needs One Owner for Audio

On iOS the loop taught one more lesson. At first the player and the recorder each configured the audio session on their own. Worse, iOS suspends an app whose audio session goes idle, and in a talk the idle part was the soul's thinking. Now a single owner sets the session to play-and-record once and keeps the microphone open from the first listen until hands-free ends, and the player leaves the session alone while that owner holds it. The talk continues with the screen off and the phone in a pocket, and one Live Activity follows the whole conversation, showing "listening…" or "speaking…" on the lock screen.

Code

The listener's phases as a pure reducer·typescript

// The hands-free loop as a pure reducer: every transition is explicit and testable.
type Phase = 'off' | 'connecting' | 'listening' | 'waiting' | 'paused' | 'stopped';

type Event =
  | { kind: 'start' }
  | { kind: 'socketOpen' }
  | { kind: 'utterance'; text: string }        // end of speech decided
  | { kind: 'replyDone'; playerBusy: boolean }  // the reply's stream ended
  | { kind: 'playerIdle' }                      // nothing playing or queued
  | { kind: 'quietMinute' }
  | { kind: 'socketClosed'; code: number; reason: string }
  | { kind: 'pause' }
  | { kind: 'resume' }
  | { kind: 'close' };

interface Loop { phase: Phase; reason: string | null; streamEnded: boolean }

const LISTEN_AGAIN: Loop = { phase: 'connecting', reason: null, streamEnded: false };

export function step(loop: Loop, event: Event): Loop {
  switch (event.kind) {
    case 'start':
    case 'resume':
      return LISTEN_AGAIN;
    case 'socketOpen':
      return loop.phase === 'connecting' ? { ...loop, phase: 'listening' } : loop;
    case 'utterance':
      return loop.phase === 'listening' ? { ...loop, phase: 'waiting', streamEnded: false } : loop;
    case 'replyDone':
      if (loop.phase !== 'waiting') return loop;
      // Listen again only when the reply is finished AND nothing is playing.
      return event.playerBusy ? { ...loop, streamEnded: true } : LISTEN_AGAIN;
    case 'playerIdle':
      // An idle player before the reply has ended is not a reason to listen.
      return loop.phase === 'waiting' && loop.streamEnded ? LISTEN_AGAIN : loop;
    case 'quietMinute':
      return { ...loop, phase: 'stopped', reason: 'Nothing heard for a minute; tap to listen again.' };
    case 'socketClosed':
      return { ...loop, phase: 'stopped', reason: `Listening closed (${event.code}): ${event.reason || 'no reason given'}` };
    case 'pause':
      return { ...loop, phase: 'paused' };
    case 'close':
      return { phase: 'off', reason: null, streamEnded: false };
  }
}

let loop: Loop = { phase: 'off', reason: null, streamEnded: false };
const script: Event[] = [
  { kind: 'start' }, { kind: 'socketOpen' }, { kind: 'utterance', text: '내일 날씨 어때?' },
  { kind: 'playerIdle' },                        // too early: the reply hasn't ended
  { kind: 'replyDone', playerBusy: true }, { kind: 'playerIdle' },
  { kind: 'socketOpen' }, { kind: 'quietMinute' },
];
for (const event of script) {
  loop = step(loop, event);
  console.log(event.kind.padEnd(12), '->', loop.phase, loop.reason ?? '');
}

Exercise

Run the reducer (npx tsx loop.ts) and read the transitions. Then add two events you think are missing: the socket closing before it ever opened, and the user sending a typed message while the loop is waiting. Decide the next phase for each and add a line to the script that exercises it.

Hint

A socket that closes while still connecting used to leave the phase stuck at 'connecting' forever. It should go to 'stopped' with the close code and reason on screen, exactly like a close after opening. A typed message while waiting doesn't change the listener at all: the reply to it will end and the player will go idle, and the loop listens again from those facts.