SSE Streaming — Token-by-Token to the Browser

Why SSE over WebSockets

SSE (Server-Sent Events) is one-way: server pushes, client listens. WebSockets are two-way. For a chat where the client sends one prompt and the server streams back tokens, SSE is the right shape — simpler, plays nicer with HTTP middleware, and reconnects are free.

The wire format

SSE is just text/event-stream over HTTP. Each event is data: <payload>\n\n. The browser's EventSource API or any fetch + ReadableStream reader can parse it.

Async generator → SSE

FastAPI's StreamingResponse takes any iterable. An async generator yields strings; FastAPI flushes each one to the network. The event loop stays free to do other work between yields.

Write before show: Every chunk yielded to the browser must already be persisted in the JSONL ground truth. If the JSONL append fails, we don't yield. This is what 'never present broken chat to Dad' means in code (Truth track, lesson 1).

Code

Server side — async generator yielding SSE events·python

async def event_stream(req: ChatRequest):
    yield f"data: {json.dumps({'type':'session','id':session_id})}\n\n"
    async for chunk in adapter.stream(req.prompt):
        # eager append to JSONL ground truth — must succeed before yield
        await jsonl_logger.append(conversation_id, chunk)
        yield f"data: {json.dumps(chunk)}\n\n"
    yield f"data: {json.dumps({'type':'done'})}\n\n"

Client side — ReadableStream parser (TypeScript)·ts

async function* parseSSE(response: Response): AsyncGenerator<SSEEvent> {
  const reader = response.body!.getReader();
  const decoder = new TextDecoder();
  let buf = '';
  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    buf += decoder.decode(value, { stream: true });
    const lines = buf.split('\n\n');
    buf = lines.pop()!;
    for (const line of lines) {
      if (line.startsWith('data: ')) {
        yield JSON.parse(line.slice(6));
      }
    }
  }
}

Why SSE over WebSockets

The wire format

Async generator → SSE

Code

Progress