Lesson 10 of 10 · published

One Press, One Turn: A Wrist Microphone That Cannot Reopen

~18 min · on-the-wrist, audio, state-machine, privacy, airpods

Level 0Bundle Opener

0 XP0/84 lessons0/17 achievements

0/100 XP to next level100 XP to go0% complete

"Using hands-free on the watch just doesn't work the way it does on the phone, so I think the right thing is to cut it manually, one turn at a time."

When the Dictation Screen Is the Wrong Door

The dictation lesson's door is right for a capture: the system's screen opens, you speak, and you get text back. A spoken conversation with no hands asks for something else. watchOS offers no speech recognizer an app can drive outside that screen, there is no Speech framework on the watch, and the watch cannot reach the engine anyway. So the conversation app splits the work: the wrist records and the phone listens. The watch records with AVAudioRecorder (AAC, 16 kHz mono, metering on) and decides the end of speech itself. It spends the first half second measuring the room. Speech is anything 10 dB above that floor and louder than -45 dB, and three seconds of quiet after speech ends the utterance. Less than 0.4 seconds of speech is "nothing to send". The file goes to the phone with transferFile and a ring, the phone transcribes it in the language chosen on the wrist, and the reply comes back like any other.

A Loop Is a Door That Opens Itself

The first hands-free design copied the phone's: listen, send, hear the reply, listen again. On a wrist every way back to the microphone was a way for the wrong words in. On a morning in Seville the loop recorded and sent five turns in six minutes, and two were not the owner's. A 578-character reply was read on the phone for about two minutes. Ninety seconds after the phone said it had started reading, the watch listened again and sent her second paragraph as his turn. Later it sent a question from someone nearby. Another day it never came back from "Thinking" at all. The owner believed hands-free was off. The decision that followed was his: one turn per press.

That is a design decision, not a bug fix. A microphone that reopens on its own will eventually hear the reply it just played, a bystander, or a room it was never meant to be in. Taking the loop out removes all three at once, which no amount of tuning the loop could do.

Every End Is Idle

The turn is a small pure state machine, shared into the phone's test suite so the rules are held without a watch. A start (the Action Button's control, the complication, a tap) asks for a token. Only that token, presented when the audio session is actually up, opens the microphone. The utterance leaves for the phone and the microphone closes for good in this turn. The reply plays and the turn returns to off. Every other way a turn ends (the reply failed, the voice was late, the phone's reading ended, Stop, leaving the app) is off or stopped(reason), never listening. Every end bumps a generation, so a start still waiting on permission or audio opens nothing when it finally arrives. A start while a turn is under way is ignored, which matters on a cold launch where the control, a launch request and a URL can all ask at once. And while she thinks, a tap does nothing; the next turn is another press.

The loop's branches were deleted, not hidden behind a flag. A flag keeps the door in the code, and the next change will find it.

Off Means Off

The turn holds the app's one runtime session from start to end, so it survives the wrist going down. Leaving the app is different. When the system ends the session because the app is no longer in front, or when the app goes to the background with no session running, the turn ends: a microphone that wakes up with the app is not a conversation the wearer is in. The phone has the same duty. Stopping hands-free clears the hook that would listen again when a reading ends, so a reading that finishes after the conversation closed opens nothing.

Whose Ears Decide Who Speaks

AirPods play for one device at a time. When the owner's voice talk was open on the phone, the AirPods were the phone's, and a reply played on the watch came out of the watch's speaker instead. Now, when the phone holds a personal route (Bluetooth, wired headphones, the car), the phone reads the wrist's reply itself, tells the watch it is reading, and marks the watch's copy of the voice silent; the watch keeps it behind a Play button. The turn ends when the phone says the reading is over, a word the phone repeats until the watch stops asking, or when a fallback timer sized to the reading runs out. Two platform rules shaped the phone's half. A reading started from the background needs an app that is still running, and iOS refused one that was not ("no assertions to start mixable playback"). And a navigation prompt interrupted a .spokenAudio session, so a background reading now mixes and ducks under the prompt instead. The owner also ruled out a second route that tried to play the reply from the watch into the phone's AirPods: one road, the device that holds the ears.

The app-level story of the same decisions is in the Voice Mode quest: One Press, One Turn and The Phone Reads It to Your Ears.

Code

One voice turn as a pure state machine: only a live start opens the microphone, and every end is idle·swift

import Foundation

/// One voice turn on the wrist. Nothing opens the microphone but `audioUp` with the token a
/// `begin` handed out, and every way a turn ends is .off or .stopped, never .listening.
struct VoiceTurn: Equatable {
    enum Phase: Equatable { case off, listening, thinking, speaking, stopped(String) }

    private(set) var phase: Phase = .off
    private(set) var pending: String?          // the utterance this turn waits on
    private var generation = 0                 // every end voids every start still on its way
    private var launch: Int?

    var micOpen: Bool { phase == .listening }
    private var idle: Bool {
        if case .stopped = phase { return true }
        return phase == .off
    }

    /// A press. nil while a turn is already up or starting: on a cold launch the control,
    /// a launch request and a URL can all ask at once.
    mutating func begin() -> Int? {
        guard launch == nil, idle else { return nil }
        generation += 1
        launch = generation
        return generation
    }

    /// The audio session is up. False if the turn was stopped meanwhile: then nothing opens.
    mutating func audioUp(_ token: Int) -> Bool {
        guard launch == token, token == generation else { return false }
        launch = nil
        phase = .listening
        return true
    }

    /// The utterance left for the phone: the microphone is closed for the rest of this turn.
    mutating func sent(_ id: String) {
        guard phase == .listening else { return }
        pending = id
        phase = .thinking
    }

    mutating func replyPlaying(_ id: String) {
        guard phase == .thinking, id == pending else { return }
        phase = .speaking                      // pending keeps the id: only this reply can end the turn
    }

    /// This turn's reply played out, the phone finished reading it, or a fallback timer ran out.
    /// An older reply's end, arriving late, is not this turn's end.
    mutating func replyOver(_ id: String) {
        guard id == pending, phase == .thinking || phase == .speaking else { return }
        end()
    }

    /// Stop, Close, or leaving the app.
    mutating func stop() { end() }

    mutating func halt(_ reason: String) {
        end()
        phase = .stopped(reason)
    }

    private mutating func end() {
        generation += 1
        launch = nil
        pending = nil
        phase = .off                           // never .listening: the next turn is another press
    }
}

The wrist's own end of speech: calibrate the room, then wait for three quiet seconds·swift

import Foundation

/// Fed one meter reading (in dB) every 100 ms while the turn records.
struct EndOfSpeech {
    var quiet: TimeInterval = 3          // silence after speech that ends the utterance
    var calibrate = 5                    // the first half second measures the room
    var margin = 10.0                    // speech is this far above the room...
    var minimum = -45.0                  // ...and never quieter than this

    private(set) var speechSeconds: TimeInterval = 0
    private var readings: [Double] = []
    private var floor: Double?
    private var lastSpeech: TimeInterval?
    private var lastReading: TimeInterval?

    /// true when the utterance is over. Under 0.4 s of speech, the caller sends nothing.
    mutating func feed(_ db: Double, at t: TimeInterval) -> Bool {
        let step = lastReading.map { max(0, t - $0) } ?? 0
        lastReading = t
        guard let room = floor else {
            readings.append(db)
            if readings.count >= calibrate { floor = readings.sorted()[readings.count / 2] }
            return false
        }
        if db > max(room + margin, minimum) {
            speechSeconds += step
            lastSpeech = t
            return false
        }
        floor = room * 0.95 + db * 0.05     // quiet readings keep the floor honest as the room drifts
        if let last = lastSpeech, t - last >= quiet - 0.001 { return true }   // 100 ms steps sum to 2.999...
        return false
    }
}

External links

Exercise

Put VoiceTurn in a target your phone tests reach and prove four rules. First, every ending path ends in .off or .stopped. Second, a stop() between begin() and audioUp opens nothing. Third, a second begin() during a turn returns nil. Fourth, no sequence of calls after sent makes micOpen true again. Feed EndOfSpeech a recorded meter trace and check where it ends. Then, on a real watch, finish one turn and let a radio or a nearby conversation play for two minutes. Confirm that nothing was recorded or sent.

Hint

A property test helps with the fourth rule: generate random sequences of the mutating calls and assert that micOpen never becomes true without a fresh begin and audioUp. In the device check, read the phone's log for incoming files, not only the watch's screen.