Client Initialization

~22 min · client, async, httpx

Level 0Tokenizer

0 XP0/54 lessons0/10 achievements

0/120 XP to next level120 XP to go0% complete

The SDK provides two client classes: OpenAI() for synchronous code and AsyncOpenAI() for async/await patterns.

Async Client

Key config options: timeout (default 600s), max_retries (default 2), base_url (for proxies/Azure), organization, project. You can override per-request with client.with_options(max_retries=5, timeout=120.0).

Why one client per process

Each call to OpenAI() constructs an httpx client underneath. httpx clients hold a connection pool. Constructing per-call means creating and tearing down a connection pool on every request — TLS handshake every time, no keep-alive, no pooling benefit. Under load, this manifests as inexplicable latency tail and connection-cap warnings.

The fix is one line: build the client at app startup, store it on a module-level variable or DI container, reuse forever. Per-call with_options() overrides give you per-request tuning without rebuilding the pool.

Code

Sync OpenAI() client·python

from openai import OpenAI

# Minimal — reads OPENAI_API_KEY automatically
client = OpenAI()

# Full configuration
client = OpenAI(
    api_key=os.environ["OPENAI_API_KEY"],
    organization="org-XXXXXXXXXXXXXXXX",
    project="proj_XXXXXXXXXXXXXXXX",
    timeout=60.0,       # default is 600s (10 min)
    max_retries=2,      # default is 2
    base_url="https://api.openai.com/v1",  # override for proxies
)

AsyncOpenAI() with custom timeout·python

from openai import AsyncOpenAI

async_client = AsyncOpenAI(
    api_key=os.environ["OPENAI_API_KEY"],
)

# Use in async context
async def main():
    response = await async_client.responses.create(
        model="gpt-5.4",
        input="Hello!",
    )
    print(response.output_text)

External links

Exercise

Build a tiny FastAPI app with a single GET /chat endpoint that uses AsyncOpenAI. Verify with an Apache Bench / hey run that 100 concurrent requests don't block each other.