Skip to content
C.W.K.
Stream
Lesson 02 of 06 · published

The Ear Goes to the Cloud, on Purpose

~14 min · voice-chat, realtime-stt, scribe, privacy, tradeoff

Level 0Unlit
0 XP0/48 lessons0/13 achievements
0/100 XP to next level100 XP to go0% complete
"Dictation keeps the rule. The conversation crossed it — and wrote down why."

The Line This Quest Drew

Three tracks of this quest lean on one rejection: no cloud transcription. Dictation still honors it to the letter. WhisperKit or Apple's SpeechAnalyzer hears you on the Mac, and dictation audio never leaves as a file and is discarded once transcribed, apart from the bounded failure rescue track 8 describes. Voice chat does not honor it. When Dad opens a talk, the microphone streams his voice to ElevenLabs' Scribe v2 Realtime, a cloud model. That was his choice, made with the tradeoff in view, and the quest should say so plainly instead of letting "your voice never leaves your Mac" stand as if it covered every surface.

How the Audio Travels

Nothing is relayed. cwkBellows, the family's voice engine, holds the ElevenLabs account key and mints a single-use token. cwkPippa builds the complete socket address around that token: the language, the end-of-speech window, and keyterms — the names of the souls this soul may see, in Korean for a Korean session. Firekeeper's listener (RealtimeListener in the voice kit) then connects straight to ElevenLabs. The key never touches the Mac, and the audio passes through no server of ours on its way. The stream opens only after Dad opens a talk; between talks the microphone is closed.

Firekeeper  --POST /api/stt/realtime-session-->  cwkPippa
cwkPippa    --POST /api/stt/realtime-token---->  cwkBellows (holds the key)
cwkPippa    <-- full socket URL: token + language + VAD window + keyterms
Firekeeper  ==== mic audio, live ============>  ElevenLabs Scribe v2 Realtime
            <=== partial words, commits =====

Dictation never takes this path. Its ear stays on the Mac.

Why the Conversation Crossed the Line

The decision is recorded with its reasons. A talk needs partial words while Dad is still speaking, an end-of-speech window that cwkPippa's voice settings define (3 seconds by default, tunable per device), and keyterms so names come out right — 피파 was being transcribed as FIFA in 30 of 682 dictated turns before the Korean soul names rode the session. It also keeps dictation's local STT and cleanup lanes entirely out of the talk, so neither can slow or break the other. And the family runs one transcription vendor everywhere, so the WebUI, the phone and Firekeeper hear Dad the same way. Every one of those reasons is about the conversation. None of them applies to dictating into a text field, which is why dictation did not move.

A Commit Is Not the End of Speech

Track 3 taught how to fake streaming on top of a batch model. Here the model streams natively, and it has its own edges. Scribe commits a segment on its own after about 35.8 seconds of continuous talk, voice activity or not. The first loop took every commit as the end of the utterance, and Dad's demo went out mid-sentence. Now a commit is asked whether silence followed it: if a third or more of the window was loud, he is still talking, the piece is kept, and listening continues. After Scribe's quiet window, the loop also waits out the rest of Dad's window; if two chunks in a row come back loud (above three times the committed silence's floor and above 300 RMS), the turn stays open, and words that arrive within 2.5 seconds join it.

When the Realtime Model Drops Words

The second edge is worse because it is silent. On the WebUI, a 132-second count reached the text with 20 seconds of it missing: the realtime model had stalled inside a segment. The batch model wrote every word of the same recording in 3.3 seconds. So an utterance of 30 seconds or more is sent once more, as a file, to cwkPippa's batch transcription (/api/stt, never asked to keep the audio), and the batch text replaces the live words. If that call fails, the live words still go out. The direction of track 3 flips: there the batch model was the base and streaming was the illusion; here streaming is the base and batch is the ground truth you fall back on when it matters.

Scope a rule to the surface whose reasons produced it. "No cloud STT" was argued from dictation: an input method, used all day, into every app. A conversation has different needs and a different owner of the tradeoff. Keep the rule where its reasons hold, cross it where they do not, and write the crossing down so nobody mistakes it for drift.
Say which surface a privacy promise covers. "Your voice never leaves your Mac" is true of Firekeeper's dictation and false of its voice chat. A promise without its scope becomes a false statement the day a second surface ships.

Code

Closing an utterance in a talk (VoiceChatController with the voice kit's rules)·text
on commit(segment):
  if loud share of Scribe's window >= 1/3:
      keep the piece, keep listening          # he is still talking
  else:
      start Dad's extra wait
      SpeechResumeWatch armed:
        2 loud chunks in a row (> 3x floor, > 300 RMS) -> hold the turn
        words within 2.5 s                           -> join the same turn

on utterance end:
  if duration >= 30 s:
      batch = POST /api/stt (multipart, no keep_audio, 45 s timeout)
      send(batch.text ?? live words)          # batch is the ground truth
  else:
      send(live words)

External links

Exercise

Take a privacy rule you hold for one feature ("this data never leaves the device"). List the reasons behind it. Now imagine a second feature that wants to break it. For each reason, mark whether it still applies to the second feature. If most do not, write the one-sentence scope the rule should have had from the start.
Hint
Rules argued from one surface tend to be stated as universal. The fix is not to abandon the rule; it is to name the surface it protects. Then a second surface that crosses it is a recorded decision with its own reasons, not a quiet erosion of the first.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.