Say It, Then Do It

"He heard 'I'll turn it on' after the air conditioner was already on. The words were right. Their order in time was wrong."

The Bug

In a spoken turn, Pippa says a short line before she uses a tool — "hold on, I'll turn it on" — and then the tool runs. The first panel waited for the whole streamed answer, then read it. So Dad heard the announcement after the action had already happened in the room. Nothing was technically lost; the conversation just stopped making sense, because speech is a promise about what comes next, and this promise arrived late.

Cut the Reply at Every Tool Call

The fix borrows the WebUI's rule and lives in the voice kit as ReplyUnits. While the reply streams, the text so far becomes a unit the moment a tool call begins — at its tool_input_start event, or at its tool_use when no start arrived. That unit is synthesized and read at once, while the stream keeps going. At the end of the answer, only the rest after the last cut is read. A stretch with nothing speakable in it (only cards or tags) is passed over. A stream_reset stops the reading, drops the queue and starts the cuts over.

stream:  "잠깐, 켜 볼게" | tool_use(a home device) | "켰어."
          ^ unit 1 read NOW        ^ waits for quiet   ^ unit 2 at the end

phase:   speaking ........ thinking ................ speaking -> listening

One Unit at a Time, Never Split

Units are read one at a time, in order, and a unit is never split to start sooner — the voice engine's rule, so a sentence keeps its prosody. The panel says speaking while a unit plays or waits, goes back to thinking between the line before a tool and the rest of the answer, and listens again only after the last unit of a finished answer. If an answer breaks off, nothing more is spoken, but lines already cut before the break were whole units and are read.

The Speech Gate: the Tool Waits for the Ear

Reading the line first is half of it. The other half is making the tool wait until Dad has actually heard the line. Queuing a line before a tool reports {"busy": true} to cwkPippa's speech gate (POST /api/conversations/{id}/playback). The queue draining reports false, and so do a cut-in, a pause, a stop, a leave and a reset. An action tool on the engine — a home device, an On Air effect — waits for quiet, up to 12 seconds. The reports go out in order and best effort, and the talk never awaits them: a slow server can delay an action, never the voice.

Cutting In Across Units

With several units in flight, "how far did he hear?" needs care. A key cut, a face click or a voice cut-in stops the whole queue. cwkPippa is told the playing unit's start plus how far into it the audio got, or the end of the last unit read through. If the cut lands while the reply still streams, the rest is passed over, the stream is followed quietly to its end so the report can reach the reply's id, and Dad's next turn in that conversation waits for that report. The record of what he heard stays true even when he interrupted.