Fallback Chains

A FallbackAdapter tries each model in sequence, falling back on rate limits (429) or server errors (503).

The fallback chain is essential for production reliability. If the primary model is rate-limited or experiencing downtime, the agent automatically degrades to a capable alternative rather than failing entirely.

Different failure modes, different paths

cwkPippa's brain fallback is Codex → Claude → Gemini per task, with explicit per-task overrides. Why three providers? Each is degraded ~4h/day in different windows; the chain absorbs it. Why not just one with retries? Because the failure isn't random — when Claude's having a bad hour, retrying Claude five times is five guaranteed failures.

The 2-of-2 pattern within a single provider also matters: native vision + Read-tool fallback (cover model degradation), OAuth + API-key (cover identity-layer outage), SQLite + JSONL (cover store corruption). Each pair is deliberate. 'Cleaning up' one path because it 'looks duplicate' creates a single point of failure.

Code

Two-of-two: native vision + Read-tool fallback·python

class FallbackAdapter(LLMAdapter):
    """Try each adapter in order, falling back on 429/503 errors."""
    def __init__(self, adapters: list[tuple[LLMAdapter, str]]):
        self.adapters = adapters  # (adapter, model_name) pairs

    async def stream(self, messages, tools=None, model=None):
        last_exc = None
        for adapter, model_name in self.adapters:
            try:
                async for chunk in adapter.stream(messages, tools, model or model_name):
                    yield chunk
                return  # Success — stop trying
            except httpx.HTTPStatusError as e:
                if e.response.status_code in (429, 503):
                    logger.warning(f"Falling back from {model_name}: HTTP {e.response.status_code}")
                    last_exc = e
                    continue
                raise  # Non-retriable error — propagate
        raise last_exc

# Usage:
fallback = FallbackAdapter([
    (OpenAIAdapter(model="gpt-5.4"), "gpt-5.4"),      # primary
    (OpenAIAdapter(model="gpt-4.1"), "gpt-4.1"),      # secondary
    (OpenAIAdapter(model="gpt-4o-mini"), "gpt-4o-mini"), # tertiary
])

Different failure modes, different paths

Code

External links

Exercise

Progress

Comments 0