httpx Basics

~22 min · httpx, client, timeouts

Level 0Tokenizer

0 XP0/54 lessons0/10 achievements

0/120 XP to next level120 XP to go0% complete

httpx is a modern, async-capable HTTP client for Python. It's what the official OpenAI SDK uses internally.

Production AsyncClient Setup

Key insight: Do NOT create a new AsyncClient per request. Connection setup is expensive. Create one client and reuse it throughout the application lifecycle. Use context managers for proper cleanup.

The two-class lifecycle

httpx exposes httpx.Client (sync) and httpx.AsyncClient (async). Both are context managers (with httpx.Client() as client, async with httpx.AsyncClient() as client). Both pool TCP connections. Constructing per-request defeats the pool and kills throughput.

For LLM streaming, configure timeouts explicitly: connect=10 (don't wait forever for a flaky DNS resolution), read=300 (long, because streamed responses can take minutes), write=10, pool=10. timeout=None is dangerous — a hanging response turns into a leaked connection that lives until the OS reclaims the socket.

Code

Sync httpx.Client with timeouts·python

import httpx

client = httpx.AsyncClient(
    base_url="https://api.openai.com/v1",
    headers={
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json",
    },
    timeout=httpx.Timeout(
        connect=10.0,   # TCP connection timeout
        read=300.0,     # Response read timeout (long for streaming)
        write=30.0,     # Request upload timeout
        pool=5.0,       # Connection pool acquisition timeout
    ),
    limits=httpx.Limits(
        max_keepalive_connections=20,  # Idle connections
        max_connections=100,           # Max total connections
        keepalive_expiry=30.0,         # Seconds before closing idle
    ),
    http2=True,  # Enable HTTP/2 multiplexing
)

External links

Exercise

Build a get_or_none(url, *, client) helper using httpx.AsyncClient with the timeouts above. Use it to fetch 10 URLs concurrently with asyncio.gather; one of them should be a known-slow URL. Verify the timeout fires cleanly.