The Reply Mic and the Chosen Ear

The other half of the conversation

A card gets an answer. The server asks something back, names a price, or says the kitchen is out of what Dad wanted. Under the composer on Spark's conversation screen sits the reply mic. It has three parts: a caption that reads Their reply in, a row of language chips, and a wide button, Record their reply, which becomes Stop and relay while it records, with a level bar and a seconds counter beside it.

The chips start on the scene's language, the one the card was written in. Dad can switch them before he records. Nothing listens to the audio to decide what language it is. That is a design ruling: the listening language for a foreign reply is the scene's language by default, switchable by Dad, and never silently detected. The web client's language list carries the same note, chosen and never guessed, and traces it to Dad's earlier ruling on dictation. There are ten choices, from Spanish, Catalan, Basque and Galician to Japanese and Chinese.

The phone records with the kit's VoiceClipRecorder, and the web uses the browser's MediaRecorder. Both hand the engine one finished clip.

Keep the recording first

The clip goes up as PUT /api/moments/{id}/replies/{media_id}. Three choices in that route matter.

The recording's id is minted by the client, a ULID made on the phone, and the client request id rides as a query parameter. If the same upload arrives twice, the route finds the earlier turn and returns it with created: false. The engine test sends one reply twice and asserts that Bellows was asked to transcribe exactly once.
The audio is kept with the moment, as media with the role recording-other, before anything else happens. If transcription fails later, the recording is still in the corpus.
language is a required query parameter that must look like es or pt: two or three lowercase letters. A reply without it is refused with 422 before the handler runs, so nothing is stored. The test that checks this is called test_the_listening_language_is_never_guessed.

Scribe hears in the chosen language

The relay turn then runs in the background (second code block). It reads the stored audio and calls SparkVoice.transcribe (first code block), which posts the file to Bellows' /api/stt with language_code set to Dad's choice. Bellows owns the provider account, and its default speech-to-text model is scribe_v2. Spark was the first sibling engine to transcribe through Bellows, so this call is still Spark's own code. The module's docstring says it moves into the kit when a second engine needs it.

transcribe accepts no language in exactly one case: the show card's interpreter in track 6, where anyone at the table may speak and Scribe names the language it heard. The reply mic is not that case. It always passes a code.

Two failures end the turn with a reason. If Bellows cannot be reached, or refuses, the turn fails with Bellows' own message. If the transcript comes back empty, the turn fails with nothing was heard in the recording, and no brain is asked. The engine test for that case asserts that the fake Pippa received zero turns. An empty transcript is not something to interpret.

Heard, then relayed

A transcript with words in it is recorded first, as a heard result: the raw text, the language it was heard in, and about pointing at the recording. Then the relay turn asks me, in the moment's thread, with the message Spark: they said this back. and the relay addendum (third code block). The addendum warns that the transcript can be wrong, so I read it for meaning. I return a relay with their words as I understood them and the Korean, two to four answer results Dad can play, and any expressions the conversation is likely to need next.

Result	Written by	An invented example
`heard`	the engine, from Scribe's transcript, uncorrected	con tarjeta o en efectivo
`relay`	me: their words corrected, the Korean, what matters	¿Con tarjeta o en efectivo? / 카드로 할까요, 현금으로 할까요?
`answer`	me: a reply Dad can play, with when it fits	Con tarjeta, por favor. / 카드로 할게요.
`expression`	me: what may come up next	¿Me da el recibo? / 영수증 주시겠어요?

Keeping both texts matters. The heard row is what the machine got, and the relay row is what I made of it. When they differ, Dad can see where I corrected the transcript. On the phone the heard line sits small and muted above my relay, labelled with the language it was heard in, so the evidence is never hidden behind my reading of it.

Principle: Store what the instrument returned before anyone interprets it. A correction can only be checked while the uncorrected version still exists.

Answers are ready cards

On the phone each answer is a row with a play button, the source, the Korean, the pronunciation and the use line. An answer is meant for the other person by its kind, so tapping it opens the same big card as Dad's own. Answers have no voice recorded in advance. The first tap asks Bellows through /api/tts/ensure in the scene's language and plays the files, and after that the cache answers. So the loop closes. Dad says something, the other side answers, and Dad already holds a few good replies, each a tap away from being shown or heard.

Voice Mode Quest covers Scribe in its track on the ear, inside a live loop that streams. Spark's reply mic is deliberately simpler: one clip, one language chosen by a person, one transcript kept.

Code

cwkSpark, backend/voice.py: transcription through Bellows in the chosen language (verbatim, lines 48-75)·python

    async def transcribe(
        self, audio: bytes, *, filename: str, content_type: str, language: str | None
    ) -> dict[str, Any]:
        """What the other person said, in the language Dad chose. ``None`` is
        the show card's interpreter only (cwkPippa todo #609, Dad's ruling
        2026-10-23): anyone at the table may speak, so Scribe names the
        language and the answer carries it (``language`` may come back
        ``None`` when Scribe names none)."""
        if language is not None and not _LANGUAGE.fullmatch(language):
            raise VoiceError("the listening language must be a code such as es or ca")
        try:
            async with httpx.AsyncClient(timeout=120.0) as client:
                response = await client.post(
                    f"{self.base}/api/stt",
                    params={"language_code": language} if language else None,
                    files={"file": (filename, audio, content_type)},
                )
        except httpx.HTTPError as exc:
            raise VoiceError(f"cwkBellows unreachable ({self.base})") from exc
        if response.status_code != 200:
            try:
                detail = response.json().get("detail")
            except ValueError:
                detail = response.text[:200]
            raise VoiceError(f"cwkBellows could not transcribe (HTTP {response.status_code}: {detail})")
        data = response.json()
        heard_in = language or (str(data.get("language_code") or "").strip().lower() or None)
        return {"text": str(data.get("text") or "").strip(), "language": heard_in, "model": data.get("model")}

cwkSpark, backend/turns.py: the relay turn transcribes, fails with a reason, and keeps what was heard (verbatim, lines 1119-1146)·python

            audio = await asyncio.to_thread(path.read_bytes)
            listening = str(turn.get("language") or "")
            try:
                heard = await self.voice.transcribe(
                    audio,
                    filename=media["filename"],
                    content_type=str(media.get("content_type") or "application/octet-stream"),
                    language=listening,
                )
            except VoiceError as exc:
                await self._fail(turn, str(exc))
                return
            if not heard["text"]:
                await self._fail(turn, "nothing was heard in the recording")
                return
            await self._write(
                self.corpus.add_result,
                moment["id"],
                turn_id=turn["id"],
                target_language=moment.get("target_language") or "ko",
                fields={
                    "kind": "heard",
                    "source": heard["text"],
                    "language": listening,
                    "about": media["media_id"],
                    "source_key": source_key(heard["text"]),
                },
            )

cwkSpark, backend/turns.py: the relay addendum (verbatim, lines 383-407)·python

def relay_addendum(moment: dict[str, Any], listening: str) -> str:
    scene = language_name(moment.get("scene_language")) or language_name(listening) or "their language"
    heard = language_name(listening) or listening
    return "\n".join(
        [
            _spark_frame(moment),
            "",
            "### This turn: what they said back",
            "",
            f"The person Dad is talking to just answered. The words below were transcribed",
            f"from {heard}; the transcript can be wrong, so read it for meaning. Tell Dad",
            "what they said, say what matters in it, and give him answers he can play.",
            "",
            "Return ONE fenced block whose info string is `spark`:",
            "",
            "```spark",
            '{"type": "relay", "source": "<what they said, corrected>", "translation": "<in Korean>", "note": "<what matters in it>"}',
            '{"type": "answer", "source": "<a reply in ' + scene + '>", "translation": "<Korean>", "pronunciation": "<in Hangul>", "use": "<when it fits>"}',
            '{"type": "expression", "source": "...", "translation": "...", "pronunciation": "...", "use": "..."}',
            "```",
            "",
            "Give two to four answers, then any expressions the conversation is likely to",
            "need next. Outside the block, a sentence or two to Dad at most.",
        ]
    )

Exercise

Build a reply endpoint for a two-person translation tool. The client records a clip and uploads it with a client-made id and a required two- or three-letter language code. The server stores the audio first, transcribes it with any speech-to-text service you have (or a stub), fails with a clear reason on an empty transcript without calling the language model, and keeps the raw transcript as its own record before the model's corrected relay. Test that a duplicate upload is transcribed once and that a missing language is refused.

Hint

Validate the language in the route signature, so the request is refused before anything is stored. Look up the client request id first and return the earlier turn on a repeat. In the tests, count calls on your fake transcriber and your fake model: called once, and never called, are the two assertions that matter.

The other half of the conversation

Keep the recording first

Scribe hears in the chosen language

Heard, then relayed

Answers are ready cards

Code

External links

Exercise

Progress

Comments 0