Draft Fast, Refine Right

Two readers, two clocks

At a table, the two people reading the card wait differently. Dad can wait a few seconds for a better sentence. The person across from him cannot: they just spoke, and a card that shows nothing looks broken. So every utterance gets two passes. A fast draft streams onto the card word by word, then a refined version replaces it. The design's latency budget, counted from the end of speech, aims for the first words within about three seconds and a complete draft by about five, with Pippa's refined version following while the draft is already readable.

그래서 멍때리면 안되고 UI에서 진행상황을 계속 보여줘야 해. 실제로 느려서 느리게 느끼는 게 아니라 UI가 멈춰 있어서 느린 걸로 보이는 거니까.

Dad's second line of the same ask: so it must never zone out, and the UI has to keep showing progress, because things feel slow when the UI freezes, more than when they actually are slow.

The fast draft

The draft runs on the brain assigned to Spark's live lane, the same slot The Live Lane track showed earning its place by measurement. It goes through cwkPippa's sibling utility door in its streaming form, /api/sibling/utility/stream, which answers as server-sent events: delta pieces in order, then one done carrying the whole envelope. Spark's client for that stream lives in Spark's own pippa.py, because Spark is its only caller; the family rule is that shared code moves into the kit at its second caller, never its first (Sibling Kit Quest tells that story).

Each piece is published on the sitting's event stream as exchange.delta, carrying both the new piece and the whole draft so far. When the stream ends, the whole is kept as a draft result: what was heard, what was rendered, the direction, the brain and model, and elapsed_ms, so how long the draft took is part of the record.

The draft is allowed to fail. With no brain assigned to the live lane, the engine publishes exchange.draft_skipped with that reason and moves on. A stream that errors before writing anything, or comes back empty, does the same, with its own reason. A stream that errors after some pieces arrived is not skipped: what it wrote so far is kept as the draft, the error goes unrecorded, and the refine corrects the partial text like any other draft. Two tests hold the line: no live lane skips the draft and says why, and a failed draft still reaches the refined version.

The refined version

Before the second pass starts, the engine announces exchange.refining, naming the brain, so the card can show what it is waiting on. The refine is one call through the same door's non-streaming form, on the brain of Spark's practical slot, tagged spark-exchange-refine. Its answer is one JSON object: a card when Dad spoke (the sentence in their language, his meaning in Korean, a language tag), or a relay when they spoke (their words corrected, natural Korean for Dad, an optional note).

The refined result is recorded about the draft it replaces, so the record says which draft it corrects. The card does not need that field to swap. The kit's reducer files every result under its exchange by turn_id: the draft and then the card or relay of one utterance share a turn, so the refined text takes the draft's place in that exchange. Where about does steer the card is the voice, which is recorded about the refined card and found by that id. A card made from Dad's words is then voiced in Pippa's voice, and the turn only finishes once that voice is recorded, the same rule Speaking for Dad taught for typed cards. A relay is for Dad's eyes and gets no voice.

The refine prompt ends with an instruction aimed at the screen as much as the language: keep the draft's words where they are right, and change only what the sitting corrects, so the card changes only where it must.

Lighting what changed

That instruction has a partner on the card. The kit's wordDiff splits both texts on whitespace, folds case and punctuation, and runs a longest common subsequence. Every refined word outside that subsequence is lit for a moment. Take an invented draft, ¿Me trae agua?, refined into ¿Me trae un vaso de agua, por favor?. The words kept in order are me, trae and agua, so un, vaso, de, por and favor glow, and Dad sees at a glance what Pippa added. With no draft there is nothing to compare, and nothing is lit.

Principle: A revision the reader cannot locate reads as a second, unrelated answer. Ask the second pass for small changes, then show exactly where they are.

Every wait is an event

Event	What the card can show
`exchange.transcribing`	The engine is transcribing a kept recording (the socket had dropped)
`exchange.direction`	The direction chip, before any translation
`exchange.delta`	The draft, growing word by word
`exchange.draft_skipped`	Why there is no draft
`exchange.refining`	Pippa refining, with a clock that ticks
`result.add`	The draft, then the card or relay (swap and light), then the voice
`turn.finish`	Done, or failed with its reason

The kit turns these into card state with a pure reducer, and each wait it enters stamps since, so no label is ever frozen. Spark's event stream starts with a snapshot of the whole moment and sends a keepalive every 15 seconds, so a card that reconnects rebuilds every exchange from the record instead of from memory.

On the first build, measured through the door: the direction in about 0.1 s, first draft words in about 2 s (one cold start took 6.7 s), the refined version about 11 to 16 s after the utterance was posted. The refine then still ran as a conversation turn, and the next lesson takes that apart.

Code

cwkSpark, backend/turns.py: direction at once, draft, then the announced refine about the draft (verbatim, lines 1205-1217)·python

            self.corpus.publish(moment["id"], {"type": "exchange.direction", "turn_id": turn["id"], **direction})
            draft = await self._exchange_draft(turn, moment, direction, heard, live_picks)
            moment = await asyncio.to_thread(self.corpus.moment, moment["id"]) or moment
            self.corpus.publish(
                moment["id"],
                {"type": "exchange.refining", "turn_id": turn["id"], "brain": picks["brain"]},
            )
            # The refined result is about the draft it replaces (the card swaps
            # it in and lights the words that changed).
            refining = {**turn, "target_result": draft["id"]} if draft else turn
            outcome = await self._exchange_refine(
                refining, moment, direction, heard, draft and draft.get("rendered"), picks
            )

cwkSpark, backend/turns.py: the draft streamed piece by piece, or skipped with a reason (verbatim, lines 1347-1370)·python

        loop = asyncio.get_running_loop()
        started_at = loop.time()
        written = ""
        envelope: dict[str, Any] = {}
        try:
            async for event in self.pippa.utility_stream(body):
                if event["type"] == "delta" and event.get("text"):
                    written += str(event["text"])
                    self.corpus.publish(
                        moment["id"],
                        {"type": "exchange.delta", "turn_id": turn["id"], "text": event["text"], "draft": written},
                    )
                elif event["type"] == "done":
                    envelope = event
        except PippaError as exc:
            envelope = {"ok": False, "error": str(exc)}
        rendered = str(envelope.get("result") or written).strip()
        if not rendered:
            self.corpus.publish(
                moment["id"],
                {"type": "exchange.draft_skipped", "turn_id": turn["id"],
                 "reason": str(envelope.get("error") or "the fast draft came back empty")},
            )
            return None

cwkSpark, frontend/src/kit/kitInterpreter.ts (vendored from cwkSiblingKit): which refined words to light (verbatim, lines 243-271)·typescript

/** The refined text's words, each marked when the draft did not have it in
 * that place (a longest common subsequence over whitespace-separated words,
 * case- and punctuation-folded). Separators stay attached to the word before. */
export function wordDiff(before: string, after: string): DiffToken[] {
  const split = (value: string) => value.match(/\S+\s*/g) ?? [];
  const fold = (word: string) => word.trim().toLowerCase().replace(/[\p{P}\p{S}]/gu, '');
  const old = split(before).map(fold);
  const words = split(after);
  const next = words.map(fold);
  const rows = old.length + 1;
  const cols = next.length + 1;
  const table: number[] = new Array(rows * cols).fill(0);
  for (let i = old.length - 1; i >= 0; i -= 1) {
    for (let j = next.length - 1; j >= 0; j -= 1) {
      table[i * cols + j] = old[i] === next[j]
        ? table[(i + 1) * cols + j + 1] + 1
        : Math.max(table[(i + 1) * cols + j], table[i * cols + j + 1]);
    }
  }
  const kept = new Array<boolean>(next.length).fill(false);
  let i = 0;
  let j = 0;
  while (i < old.length && j < next.length) {
    if (old[i] === next[j]) { kept[j] = true; i += 1; j += 1; } else if (table[(i + 1) * cols + j] >= table[i * cols + j + 1]) i += 1;
    else j += 1;
  }
  // With no draft there is nothing to compare: nothing is lit.
  return words.map((word, index) => ({ text: word, changed: before.trim() !== '' && !kept[index] }));
}

Exercise

Take any feature of yours that waits on a model for more than two seconds. Split it into a fast pass that streams and a slower pass that refines, and record the refined result with a pointer to the draft it replaces. Then list every waiting state the user can be in and give each one an event name and a start time. Finally, implement a word diff between draft and refinement and show the changed words.

Hint

Both passes must be allowed to fail on their own: a missing draft should never block the refine, and a failed refine should leave the draft readable. If your list of waiting states has a state with no event, that is where your UI will look frozen.

Two readers, two clocks

The fast draft

The refined version

Lighting what changed

Every wait is an event

Code

External links

Exercise

Progress

Comments 0