Rate Limits & Fallback

429 is normal, not exceptional

You will hit 429s. Not because you did something wrong, but because rate limits exist. The right reaction is exponential backoff with jitter, not panic. The wrong reaction is to retry tightly (which makes it worse) or to surface the error directly to the user.

Fallback chains across models

If Pro is rate-limited, Flash usually isn't. If Flash is, Flash-Lite usually isn't. Build a chain that walks down the cost ladder on rate-limit failures.

Fallback chains across providers

Cross-provider fallback (Gemini → OpenAI → Claude) is the next layer. Same shape: try, on failure switch, log, surface.

Safety blocks aren't retries

finishReason: SAFETY means the model filtered the output. Retrying the same prompt will get the same result. Either rephrase, loosen safety thresholds (only if the use case warrants), or surface a graceful message to the user.

Code

Resilient single call with backoff·python

import asyncio, random
from google.genai import errors

async def resilient_generate(client, model, contents, max_retries=4):
    for attempt in range(max_retries):
        try:
            return await client.aio.models.generate_content(
                model=model, contents=contents,
            )
        except errors.ClientError as e:
            if e.code == 429:
                # Rate limit — exponential backoff with jitter
                base = 2 ** attempt
                sleep = base + random.uniform(0, base * 0.5)
                print(f'[429 on {model}] sleeping {sleep:.1f}s')
                await asyncio.sleep(sleep)
                continue
            # 4xx other than 429 — don't retry
            raise
        except errors.ServerError:
            await asyncio.sleep(min(2 ** attempt, 30))
            continue
    raise RuntimeError(f'Gave up after {max_retries} retries')

Multi-model fallback chain·python

MODEL_CHAIN = [
    'gemini-2.5-pro',
    'gemini-2.5-flash',
    'gemini-2.5-flash-lite',
]

async def generate_with_fallback(client, contents):
    last_error = None
    for model in MODEL_CHAIN:
        try:
            return await client.aio.models.generate_content(
                model=model, contents=contents,
            ), model
        except errors.ClientError as e:
            if e.code == 429:
                last_error = e
                continue  # Try next model
            raise  # Other 4xx — don't shop around
        except errors.ServerError as e:
            last_error = e
            continue  # 5xx — try next

    raise RuntimeError(f'All models exhausted. Last error: {last_error}')

Handle SAFETY explicitly·python

candidate = response.candidates[0]
if candidate.finish_reason.name == 'SAFETY':
    # Surface what got blocked, for logging
    for rating in candidate.safety_ratings:
        if rating.probability.name in ('HIGH', 'MEDIUM'):
            log.warning(f'Safety block: {rating.category.name} -> {rating.probability.name}')

    # Don't retry. Surface gracefully.
    return 'Sorry, I can't help with that request.'

Cross-provider fallback (sketch)·python

ADAPTER_CHAIN: list[ModelAdapter] = [
    GeminiAdapter(api_key=GEMINI_KEY),
    OpenAIAdapter(api_key=OPENAI_KEY),     # if you have one
    OllamaAdapter(model='llama3.1:70b'),   # local fallback
]

async def stream_with_provider_fallback(messages):
    for adapter in ADAPTER_CHAIN:
        try:
            async for chunk in adapter.generate_stream(messages):
                yield chunk
            return
        except Exception as e:
            # Toast it visibly — never silent
            print(f'[{adapter.name()} failed: {e}. Trying next provider.]')
            continue
    raise RuntimeError('All providers exhausted')

Exercise

Build the multi-model fallback chain from the second code block. Force a rate-limit (call Pro 100× in a tight loop in a fresh project that hasn't built up quota). Confirm the chain falls through to Flash, then to Flash-Lite, with appropriate logging. Bonus: add per-model cooldown timers so once a model is rate-limited, you skip it for 60 seconds without trying.