Retry Posture and Reliability Patterns

Three failure classes, three responses

Claude API failures fall into permanent (4xx — your bug, do not retry), transient (5xx, 529 — retry with backoff), and rate-limited (429 — retry per retry-after). The SDK handles auto-retry up to its budget; beyond that you decide.

Circuit breakers for upstream outages

If Claude's API is degraded, retrying immediately makes things worse. A circuit breaker that opens after N consecutive failures, holds open for a cool-down, then half-opens to test recovery, prevents your service from amplifying outage. Pair with a fallback (Haiku, Sonnet on Bedrock, queued retry).

cwkPippa lives with about 4 hours/day of degraded

Real-world Claude availability is high but not perfect. cwkPippa's memory rule: ~4h/day of degraded service is normal. The fallback chain (Codex → Claude → Gemini) is a v1 must-have, not a v2 nice-to-have. Production designs assume the upstream will fail; the question is how gracefully.

Principle: Retries do not fix unreliability. Circuit breakers and fallback paths fix unreliability. Retries fix transient blips.

Code

Simple circuit breaker·python

import time

class Breaker:
    def __init__(self, threshold=5, cool_down=30):
        self.fails = 0
        self.opened_at = 0
        self.threshold = threshold
        self.cool_down = cool_down

    def allow(self) -> bool:
        if self.fails < self.threshold:
            return True
        return (time.time() - self.opened_at) > self.cool_down

    def record(self, ok: bool):
        if ok:
            self.fails = 0
        else:
            self.fails += 1
            if self.fails == self.threshold:
                self.opened_at = time.time()

breaker = Breaker()

def call_with_breaker(*args):
    if not breaker.allow():
        return fallback(*args)
    try:
        out = client.messages.create(*args)
        breaker.record(True)
        return out
    except Exception:
        breaker.record(False)
        raise

Provider fallback chain·python

def call_with_fallback(messages):
    try:
        return claude_sonnet(messages)
    except (RateLimitError, InternalServerError):
        try:
            return claude_haiku(messages)  # cheaper Anthropic model
        except Exception:
            return openai_gpt(messages)  # cross-provider last resort

Exercise

Add a fallback path to one critical Claude call (downgrade model, switch provider, or queue for batch). Force the primary to fail and watch the fallback run. Document the SLA you commit to in the user-facing contract.

Hint

Fallbacks that have never run in anger usually have a bug. Schedule a quarterly drill where you force the primary off.

Retry Posture and Reliability Patterns

Three failure classes, three responses

Circuit breakers for upstream outages

cwkPippa lives with about 4 hours/day of degraded

Code

External links

Exercise

Progress

Comments 0