Building a Token Renderer

The "append-as-you-go" UI pattern is the standard for streaming token display. Here's how to build a robust token renderer that handles partial responses, cancellation, and errors.

Core Pattern

In web applications, you'd replace sys.stdout.write with DOM manipulation — appending text to an element. For React, update state incrementally. The key is flushing output immediately rather than buffering.

The renderer is the contract between SSE and your UI

A bad renderer concatenates every token into a state variable and triggers a UI re-render on every chunk — at 50 tokens/sec, that's 50 re-renders per second, and React's reconciliation chokes. A good renderer batches into a frame budget (16ms) and only re-renders on a requestAnimationFrame tick.

Cancellation is the second contract. The user clicks 'Stop'. The renderer must (1) close the SSE stream so the server stops generating, (2) flush whatever's already been accumulated to the UI, (3) preserve partial output (don't blank it). cwkPippa's UI does exactly this — abort signal wired through to the SSE consumer, partial text preserved in JSONL even on abort.

Code

Token renderer with cursor + cancel·python

import sys
import signal

class TokenRenderer:
    def __init__(self):
        self.buffer = ""
        self.cancelled = False

    def on_token(self, token: str):
        """Append a token to the display."""
        if self.cancelled:
            return
        self.buffer += token
        sys.stdout.write(token)
        sys.stdout.flush()

    def on_done(self, usage=None):
        """Stream complete."""
        print()  # newline
        if usage:
            print(f"[{usage.get('total_tokens', '?')} tokens]")

    def on_error(self, error):
        """Handle streaming error."""
        print(f"\\n[Error: {error}]")

    def cancel(self):
        """Stop processing further tokens."""
        self.cancelled = True
        print("\\n[Cancelled]")

# Usage with streaming
renderer = TokenRenderer()
stream = client.responses.create(
    model="gpt-5.4", input="Tell me a story.", stream=True,
)
for event in stream:
    if event.type == "response.output_text.delta":
        renderer.on_token(event.delta)
    elif event.type == "response.completed":
        renderer.on_done()

Core Pattern

The renderer is the contract between SSE and your UI

Code

External links

Exercise

Progress

Comments 0