Streaming

Streaming delivers tokens as they're generated, dramatically reducing time-to-first-token and improving perceived performance. Set stream=True to receive a sequence of chunk objects.

Async Streaming with Responses API

To get token usage in a stream, pass stream_options={"include_usage": True}. The usage data arrives in the final chunk. Each streaming chunk contains a delta object with partial content, tool calls, or role information.

The right place to stream and the wrong one

Stream user-facing text — chat replies, document summarization the user is watching, anything where the human waits. Don't stream backend pipelines that just want a final string — the streaming overhead (per-chunk parsing, accumulator state) buys you nothing when no human eye is on the wire.

The cancel story matters too. Streaming responses hold an HTTP connection. If your UI lets the user 'stop generating', the abort signal must reach the SSE consumer (Python: stream.close() in a finally; JS: AbortController on fetch). Without cancel wiring, 'stop' just hides the response from view while the model finishes burning tokens server-side.

Code

Sync streaming with for chunk in stream·python

stream = client.chat.completions.create(
    model="gpt-5.4",
    messages=[{"role": "user", "content": "Write a haiku about autumn."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta
    if delta.content:
        print(delta.content, end="", flush=True)
print()  # newline at end

Async streaming with async for chunk in stream·python

import asyncio
from openai import AsyncOpenAI

client = AsyncOpenAI()

async def stream_response():
    stream = await client.responses.create(
        model="gpt-5.4",
        input="Describe the solar system.",
        stream=True,
    )
    async for event in stream:
        if event.type == "response.output_text.delta":
            print(event.delta, end="", flush=True)

asyncio.run(stream_response())

Async Streaming with Responses API

The right place to stream and the wrong one

Code

External links

Exercise

Progress

Comments 0