Fallback Chains and Degraded Modes

Three patterns, three blast radii

Fallback for Claude calls has three flavors. Same provider, smaller model — Sonnet → Haiku when Sonnet is rate-limited. Cross-provider — Anthropic → OpenAI → Gemini for capability-equivalent calls. Async deferral — push the call to Batch API or a queue when sync fails. Each has a different cost story and different UX implications.

Degraded mode is honest UX

If your fallback path returns a lower-quality answer, tell the user. 'Reply generated by Haiku because Sonnet was unavailable' is better than silently shipping a worse answer. cwkPippa surfaces the brain via UI badges so Dad knows when Pippa is running on the smaller model.

Drill it once a quarter

Untested fallbacks have bugs. Every quarter, deliberately disable the primary path in staging and watch the fallback run end-to-end. Bugs you find in a drill are bugs that will not bite you in production.

Principle: Reliability is layered. Fallbacks are the layers; drills are the proof.

Code

Three-step fallback chain·python

from anthropic import Anthropic, RateLimitError, InternalServerError, APIConnectionError

client = Anthropic(max_retries=2)

async def chat_with_fallback(messages):
    # 1) Try the planned tier.
    try:
        return ('sonnet', await call(client, 'claude-sonnet-4-6', messages))
    except (RateLimitError, InternalServerError, APIConnectionError):
        pass
    # 2) Same provider, smaller model.
    try:
        return ('haiku', await call(client, 'claude-haiku-4-5-20251001', messages))
    except (RateLimitError, InternalServerError, APIConnectionError):
        pass
    # 3) Cross-provider.
    return ('openai', await call_openai(messages))

Surface the fallback to the user·typescript

function ResponseBadge({ tier }: { tier: 'sonnet' | 'haiku' | 'openai' }) {
  if (tier === 'sonnet') return null;  // primary path, no badge
  const label =
    tier === 'haiku' ? 'fast mode (smaller model)' : 'fallback provider';
  return (
    <span className="text-xs text-amber-600" title={`Generated via ${tier}`}>
      {label}
    </span>
  );
}

Exercise

Add a quarterly drill to your runbook. The drill: in staging, simulate an outage the chain is built for (point the primary client's base URL at an address that refuses connections, which raises APIConnectionError, or stub its transport to return 529), run a representative request, and confirm the fallback path serves it within SLA. Do not drill with a revoked key: a 401 is a bug to fix, not an outage, and this chain deliberately lets it through. Document who is on call and what they should see.

Hint

If you cannot describe what the user sees during fallback, your degraded UX is undefined — fix that first.

Three patterns, three blast radii

Degraded mode is honest UX

Drill it once a quarter

Code

External links

Exercise

Progress

Comments 0