SSE Streaming — Token-by-Token to the Browser · Pippa Quest

Why SSE over WebSockets

SSE (Server-Sent Events) is one-way: server pushes, client listens. WebSockets are two-way. For a chat where the client sends one prompt and the server streams back tokens, SSE is the right shape: one direction, plain HTTP that middleware already understands, and nothing to negotiate. A reconnect is a different matter. The transport can come back on its own, but which events the client already holds, and whether the turn is still running on the server, are questions the application has to answer (the note below).

The wire format

SSE is just text/event-stream over HTTP. Each event is data: <payload>\n\n. The browser's EventSource API or any fetch + ReadableStream reader can parse it.

Async generator → SSE

FastAPI's StreamingResponse takes any iterable. An async generator yields strings; FastAPI flushes each one to the network. The event loop stays free to do other work between yields.

Write before show: Every chunk yielded to the browser must already be persisted in the JSONL ground truth. If the JSONL append fails, we don't yield. This is what 'never present broken chat to Dad' means in code (Truth track, lesson 1).

EV v11 · 2026-10-12: A stream's failure guards met a browser engine that stops reading. WebKit suspends a web view nobody can see: a background tab, a hidden window, a Voyager tab that is not in front. Measured on office on 2026-10-10, no timer fired and no chunk reached the page from five seconds after the view hid; the server's keepalives piled up and arrived together when it showed again. Two things broke on that wake-up. A 60-second idle timer that came due during the suspension fired before the piled-up chunks were read, so a healthy stream was aborted as a lost connection. And the recovery read the conversation once, as soon as the server answered its health check, while the turn was still running there; the stored reply landed minutes later with nothing on screen following it. Since that day lib/streamRecovery.ts counts only silence while the page is awake and visible, and after a loss it follows the turn until its stored reply arrives. Chromium keeps reading in the background, which is why Comet, a Chromium browser, never showed the bug. On the phone the same question had a network price: mobile track, lesson 8.

Code

Server side — async generator yielding SSE events·python

async def event_stream(req: ChatRequest):
    yield f"data: {json.dumps({'type':'session','session_id':session_id})}\n\n"
    async for chunk in adapter.stream(req.message):
        # eager append to JSONL ground truth — must succeed before yield
        await jsonl_logger.append(conversation_id, chunk)
        yield f"data: {json.dumps(chunk)}\n\n"
    yield f"data: {json.dumps({'type':'done'})}\n\n"

Client side — ReadableStream parser (TypeScript)·ts

async function* parseSSE(response: Response): AsyncGenerator<SSEEvent> {
  const reader = response.body!.getReader();
  const decoder = new TextDecoder();
  let buf = '';
  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    buf += decoder.decode(value, { stream: true });
    const lines = buf.split('\n\n');
    buf = lines.pop()!;
    for (const line of lines) {
      if (line.startsWith('data: ')) {
        yield JSON.parse(line.slice(6));
      }
    }
  }
}

SSE Streaming — Token-by-Token to the Browser

Why SSE over WebSockets

The wire format

Async generator → SSE

Code

Progress

Comments 0