Streaming: Time-to-First-Token Matters

Why stream

A non-streaming Messages call blocks until the entire response is generated. For long completions, that is several seconds of dead UX. Streaming returns Server-Sent Events as the model produces tokens, dropping perceived latency to time-to-first-token instead of time-to-last-token.

The event types you actually handle

The SDK abstracts the SSE wire format into typed events. The five worth knowing on day one: message_start (initial response shell), content_block_start (a new text or tool block begins), content_block_delta (incremental text), content_block_stop (the block closes), message_stop (the response is complete). Tool use streams as a separate block type with its own delta shape.

Streaming is harder than it looks

Disconnects in the middle of a stream leave you with partial output and no automatic retry. Token counts only finalize at message_stop. Tool-use blocks need to be reassembled from deltas before you can parse them. Streaming gives the user faster feedback at the cost of more application code.

Principle: Stream when the user is waiting. Buffer when the user is not. Do not stream just because you can.

Code

Streaming with the Python SDK helper·python

with client.messages.stream(
    model="claude-sonnet-4-6",
    max_tokens=2048,
    messages=[{"role": "user", "content": "Explain prompt caching in three short paragraphs."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
    final_message = stream.get_final_message()

print()
print("usage:", final_message.usage)

Hand-rolled event handling (TypeScript)·typescript

import Anthropic from '@anthropic-ai/sdk';
const client = new Anthropic();

const stream = await client.messages.stream({
  model: 'claude-sonnet-4-6',
  max_tokens: 1024,
  messages: [{ role: 'user', content: 'List three SSE pitfalls.' }],
});

for await (const event of stream) {
  if (event.type === 'content_block_delta' && event.delta.type === 'text_delta') {
    process.stdout.write(event.delta.text);
  }
  if (event.type === 'message_stop') {
    process.stdout.write('\n');
  }
}

const final = await stream.finalMessage();
console.log('usage:', final.usage);

Exercise

Build a tiny CLI that streams a Messages response and writes each delta to both stdout and a JSONL file, in that order. Kill the process mid-stream and confirm the JSONL still has every token that printed.

Hint

Open the file in line-buffered mode and call flush() after every write so a SIGTERM does not lose the last line.

Streaming: Time-to-First-Token Matters

Why stream

The event types you actually handle

Streaming is harder than it looks

Code

External links

Exercise

Progress

Comments 0