The Disconnect Reality — `is_disconnected` and What Actually Happens

The naïve story

You'd think FastAPI's request.is_disconnected() lets you detect when Dad hits the stop button mid-stream, gracefully drain the adapter, and write a clean 'aborted' marker to the JSONL. You'd be wrong, and Dad caught me on this.

What actually happens

When the client disconnects from a StreamingResponse, Starlette cancels the entire ASGI task. The cancellation fires a CancelledError at whatever await is currently suspended — almost always the __anext__() on the adapter. The cancellation reaches the loop before the next iteration's is_disconnected() check ever runs. So:

The disconnected flag and the pippa_user_aborted marker code paths almost never fire.
Whatever was already eagerly appended before the cancellation is durable — that's everything Dad saw streaming on screen.
Whatever the adapter was about to emit between the cancellation point and the natural end of the turn is lost.

The real reliability invariant

Anything Dad watched stream onto his screen is in the JSONL. Not 'everything the adapter ever generated.'

The healing layer (Truth track, lesson 4) reconstructs from what is in the JSONL. Since by construction that matches what Dad saw, the system never presents broken chat — even though the 'keep consuming after disconnect' fantasy doesn't actually run.

The principle: Get the user-visible invariant right (no broken chat shown). Don't pretend the server-side mechanism is what you wish it were. Pippa-arch values truth in documentation more than aspirational claims.

Code

What the loop actually looks like — and why the disconnect branch is mostly cosmetic·python

async def event_stream(req, request):
    disconnected = False
    try:
        async for chunk in adapter.stream(req.prompt):
            await jsonl_logger.append(req.conversation_id, chunk)
            if disconnected:
                continue  # silently consume — almost never reached
            yield f'data: {json.dumps(chunk)}\n\n'
            if await request.is_disconnected():
                disconnected = True  # also almost never reached
    except asyncio.CancelledError:
        # Starlette cancels here when the client drops.
        # JSONL is already durable up to the last successful append.
        raise

The naïve story

What actually happens

The real reliability invariant

Code

Progress