Async & Streaming

The .aio surface

Every method on client.models has an async sibling at client.aio.models. Same arguments, same return shape, but you await them. The same trick applies to client.aio.chats, client.aio.files, etc.

Streaming changes the function name

For streaming, the function name itself changes from generate_content to generate_content_stream. The non-streaming variant returns a single response; the streaming variant returns an iterator (sync) or async iterator (async) of chunks.

Why streaming matters

Time to first token (TTFT) on Flash is typically 200–400ms. Time to last token on a 500-word reply is 3–8 seconds. Streaming lets you start rendering at TTFT instead of waiting for the full reply, which feels dramatically faster even though the total time is the same.

Each chunk is a partial GenerateContentResponse

A chunk has the same shape as a full response but with only a slice of text in chunk.text. You concatenate as you go. usage_metadata appears only on the final chunk.

Code

Sync streaming·python

for chunk in client.models.generate_content_stream(
    model='gemini-2.5-flash',
    contents='Tell me a 200-word story about a coffee scale.',
):
    if chunk.text:
        print(chunk.text, end='', flush=True)
print()

Async streaming (the production pattern)·python

import asyncio
from google import genai

client = genai.Client()

async def stream_story():
    full = []
    final_usage = None
    async for chunk in await client.aio.models.generate_content_stream(
        model='gemini-2.5-flash',
        contents='Write a 300-word story.',
    ):
        if chunk.text:
            print(chunk.text, end='', flush=True)
            full.append(chunk.text)
        if chunk.usage_metadata:  # populated on final chunk
            final_usage = chunk.usage_metadata
    print(f'\n[total tokens: {final_usage.total_token_count}]')
    return ''.join(full)

asyncio.run(stream_story())

Streaming with multimodal input·python

from google.genai import types

for chunk in client.models.generate_content_stream(
    model='gemini-2.5-flash',
    contents=[
        'Describe what's in this image:',
        types.Part.from_uri(
            file_uri='gs://my-bucket/photo.jpg',
            mime_type='image/jpeg',
        ),
    ],
):
    if chunk.text:
        print(chunk.text, end='', flush=True)

Exercise

Build a small CLI that takes a prompt as argv and streams Flash's response token-by-token to stdout, then prints the total token count and the wall-clock time elapsed. Then run it twice with the same prompt and seed=0 — the texts should match, but the timing won't. Note the latency difference between TTFT and TTLT.