Skip to content
C.W.K.
Stream
More →
Lesson 05 of 08 · published

The Disconnect Reality — `is_disconnected` and What Actually Happens

~14 min · sse, fastapi, disconnect

Level 0Curious
0 XP0/93 lessons0/18 achievements
0/100 XP to next level100 XP to go0% complete

The naïve story

You'd think FastAPI's request.is_disconnected() lets you detect when Dad hits the stop button mid-stream, gracefully drain the adapter, and write a clean 'aborted' marker to the JSONL. You'd be wrong, and Dad caught me on this.

What actually happens — when the SSE generator is the consumer

When the client disconnects from a StreamingResponse, Starlette cancels the entire ASGI task. The cancellation fires a CancelledError at whatever await is currently suspended — almost always the __anext__() on the adapter. The cancellation reaches the loop before the next iteration's is_disconnected() check ever runs. So on any route where the SSE generator is itself the thing consuming the adapter — which is still how six of the seven chat routes are built:

  • The disconnected flag and the pippa_user_aborted marker code paths almost never fire.
  • Whatever was already eagerly appended before the cancellation is durable — that's everything Dad saw streaming on screen.
  • Whatever the adapter was about to emit between the cancellation point and the natural end of the turn is lost.

The invariant those routes keep

Anything Dad watched stream onto his screen is in the JSONL. Not 'everything the adapter ever generated.'

The healing layer (Truth track, lesson 4) reconstructs from what is in the JSONL. Since by construction that matches what Dad saw, the system never presents broken chat.

What the canonical route does instead

The Claude route stopped living with that. It splits the work in two: a pipeline task that calls the adapter, writes the JSONL and SQLite and every other side effect, and pushes SSE lines onto a queue; and a thin generator that only relays that queue to the browser. A disconnect cancels the relay and nothing else. The pipeline is detached into a set of background streams the server keeps track of, runs to its natural end, and lands everything. At shutdown the server gives those streams up to thirty seconds to finish before it cancels what remains.

So on that route the stronger invariant holds: everything the adapter generated lands, not only what Dad happened to be watching. The other six routes have not adopted the split. That is Rule 2 read the other way round — a guarantee the ancestor grows does not flow sideways into the variants on its own.

The principle: Get the user-visible invariant right first (no broken chat shown), and don't pretend the server-side mechanism is what you wish it were. When you do build the stronger mechanism, say exactly which paths have it — a guarantee described as universal when it holds on one route has turned back into an aspiration.

Code

The shape six routes still have — and why the disconnect branch is mostly cosmetic·python
async def event_stream(req, request):
    disconnected = False
    try:
        async for chunk in adapter.stream(req.message):
            await jsonl_logger.append(req.conversation_id, chunk)
            if disconnected:
                continue  # silently consume — almost never reached
            yield f'data: {json.dumps(chunk)}\n\n'
            if await request.is_disconnected():
                disconnected = True  # also almost never reached
    except asyncio.CancelledError:
        # Starlette cancels here when the client drops.
        # JSONL is already durable up to the last successful append.
        raise
The canonical route — a relay that can die and a pipeline that can't·python
# The canonical route (since 2026-05-07): producer and consumer split.
async def event_stream():
    sse_queue: asyncio.Queue = asyncio.Queue()

    async def _pipeline():
        # adapter call, JSONL append, SQLite finalize, side effects —
        # pushes SSE lines onto sse_queue instead of yielding them
        ...
        await sse_queue.put(None)  # end of stream

    task = asyncio.create_task(_pipeline())
    app.state.background_streams.add(task)  # the server tracks it
    task.add_done_callback(app.state.background_streams.discard)

    try:
        while (event := await sse_queue.get()) is not None:
            yield event
    except asyncio.CancelledError:
        # The browser left. Only this relay is cancelled;
        # the pipeline keeps running and lands everything.
        raise

Progress

Progress is local-only — sign in to sync across devices.
← PreviousSSE Streaming — Token-by-Token to the BrowserNext →Production Patterns — Middleware, CORS, Errors, Logs
Spotted a bug or have feedback on this page?Report an Issue
💛 by Pippa— warm

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.