Rate Limiting — 429, Retry-After, and the Algorithms Underneath
~10 min · auth-security, rate-limiting, 429, retry-after
Level 0HTTP Newbie
0 XP0/46 lessons0/12 achievements
0/120 XP to next level120 XP to go0% complete
"Rate limiting is the protocol's way of saying 'slow down without going away.' The status code is 429; the recovery information is Retry-After; the algorithm choice depends on how forgiving you want to be."
Why Rate Limit at All
Three independent reasons to limit how often clients can hit you:
Cost control. Each request costs CPU, RAM, DB queries, and potentially an upstream API call (which costs real money). An unbounded client can run up the bill in minutes.
Fairness. Without limits, one heavy user can starve others. Rate limits enforce "no one user gets more than X per minute," preserving capacity for everyone.
Abuse / DoS protection. A misbehaving or malicious client retrying in a tight loop can take you down. Rate limits draw the line at "this is no longer legitimate usage."
The 429 + Retry-After Contract
When the limit is hit, the server returns 429 Too Many Requests with:
HTTP/1.1 429 Too Many Requests
Retry-After: 60
Content-Type: application/json
{"error": {"code": "rate_limited", "message": "Try again in 60 seconds."}}
Retry-After can be a number of seconds or an HTTP-date. Either way, it's the server's contract: wait this long, then retry. Ignoring it is hostile — most servers double down on a non-respecting client by extending the limit window or banning the IP.
Many APIs also include rate-limit hint headers on EVERY response (success or failure):
X-RateLimit-Limit: 100 — the cap (per window).
X-RateLimit-Remaining: 47 — how many requests are left in the current window.
X-RateLimit-Reset: 1760000000 — Unix timestamp when the window resets.
Well-behaved clients track these and proactively back off as Remaining approaches zero. Anthropic, OpenAI, GitHub, Stripe, and Twilio all surface these headers.
The Four Common Algorithms
Fixed Window. 100 requests per minute. Window resets on the minute boundary. Simple to implement; vulnerable to the "thundering herd at the boundary" — clients can fire 100 just before reset and 100 just after, effectively 200 in 2 seconds.
Sliding Window Log. Track every request's timestamp; count requests in the last 60 seconds. Smooth and accurate; memory-heavy at scale (one entry per request).
Sliding Window Counter. Combine fixed windows with weighted overlap (count current window 100% + previous window proportionally). Approximate but cheap; what most production systems use.
Token Bucket. Each client has a bucket holding tokens. Each request consumes one. Tokens refill at a fixed rate. Allows bursts (full bucket) within a sustained rate. The most flexible; what Stripe and AWS use.
The header tells clients what's possible; the algorithm decides what's polite. Surfacing X-RateLimit-Remaining lets well-behaved clients self-regulate without ever hitting 429. The 429 + Retry-After is the safety net for clients that don't. Both are necessary; neither alone is enough.
Where to Apply Limits
Three layers of granularity, usually combined:
Per IP — protects against unauthenticated abuse. Coarse; can punish NAT'd users.
Per API key / user — fair per-account limit. The most common.
Per endpoint — expensive endpoints (search, ML inference) get tighter limits than cheap ones.
Combine: "100 req/min per API key globally, but only 10 req/min on /search." Surface both via headers when possible.
Distributed Rate Limiting
If your service runs on multiple instances, in-memory rate counters give each instance its own quota — a user can hit 5x your intended limit by hitting 5 different replicas. The standard fix: Redis as the shared counter (atomic INCR + EXPIRE per key). Adds a Redis round-trip per request; usually worth it.
For very high traffic, sample-based approaches (count one of every N requests, multiply) reduce Redis load at the cost of approximation.
cwkPippa's Rate Limiting Reality
cwkPippa doesn't currently rate-limit its own API — there are at most two human clients (Dad + Pippa via WebUI). The brain adapters DO encounter rate limits from upstream APIs (Anthropic, OpenAI), and the fallback chain (Codex → Claude → Gemini) handles those by switching brains on 429. The Gemini OAuth quota visible-fallback (see feedback_no_auto_gemini_fallback) is the canonical example: when Gemini's free quota hits 429, the system flips to paid API key automatically with a sticky toast warning. Real production: Stripe rate-limits cwkPippa's webhook receiver if we ever ship one to third parties; that's the day per-API-key limits come in.
Code
FastAPI rate limit with slowapi — per-endpoint tightness·python
# FastAPI — token-bucket rate limit using slowapi (the standard library)
from fastapi import FastAPI, Request
from slowapi import Limiter, _rate_limit_exceeded_handler
from slowapi.errors import RateLimitExceeded
from slowapi.util import get_remote_address
app = FastAPI()
limiter = Limiter(key_func=get_remote_address) # rate-limit per client IP
app.state.limiter = limiter
app.add_exception_handler(RateLimitExceeded, _rate_limit_exceeded_handler)
@app.get('/api/cheap')
@limiter.limit('100/minute')
async def cheap_endpoint(request: Request):
return {'ok': True}
@app.get('/api/expensive')
@limiter.limit('10/minute') # tighter limit on expensive endpoints
async def expensive_endpoint(request: Request):
return {'ok': True}
# Hitting the limit produces:
# HTTP/1.1 429 Too Many Requests
# Retry-After: 60
# X-RateLimit-Limit: 10
# X-RateLimit-Remaining: 0
# X-RateLimit-Reset: 1760000000
# {"error":"Rate limit exceeded: 10 per 1 minute"}
Client: respect Retry-After + pre-emptive slowdown when Remaining < 5·python
# Client side — respect Retry-After and X-RateLimit-Remaining
import httpx, time
class RateAwareClient:
def __init__(self, base_url: str):
self.client = httpx.Client(base_url=base_url)
def request(self, method: str, path: str, **kwargs) -> httpx.Response:
for attempt in range(3):
resp = self.client.request(method, path, **kwargs)
if resp.status_code == 429:
wait = int(resp.headers.get('Retry-After', 5))
time.sleep(wait)
continue
# Pre-emptive backoff if we're close to the limit
remaining = int(resp.headers.get('X-RateLimit-Remaining', 100))
if remaining < 5:
reset_ts = int(resp.headers.get('X-RateLimit-Reset', time.time() + 60))
wait = max(0, reset_ts - int(time.time()))
if wait > 0:
time.sleep(min(wait, 1)) # gentle slow-down
return resp
return resp # gave up; surface the 429 to caller
Distributed rate limit — Redis as the shared counter across replicas·python
# Distributed rate limit via Redis — token bucket with atomic check-and-decrement
import time, redis
rdb = redis.Redis()
def is_allowed(client_id: str, limit: int = 100, window_s: int = 60) -> tuple[bool, int]:
"""Returns (allowed, remaining). Uses Redis INCR + EXPIRE for atomicity."""
now = int(time.time())
window_start = now - (now % window_s)
key = f'rl:{client_id}:{window_start}'
pipe = rdb.pipeline()
pipe.incr(key)
pipe.expire(key, window_s * 2) # leeway for cross-window requests
count, _ = pipe.execute()
remaining = max(0, limit - count)
return count <= limit, remaining
# In FastAPI middleware:
# allowed, remaining = is_allowed(client_id)
# if not allowed:
# return JSONResponse(429, content={'error': 'rate_limited'},
# headers={'Retry-After': str(window_s),
# 'X-RateLimit-Remaining': '0'})
Add rate limiting to one FastAPI endpoint (/api/expensive) with a 5-per-minute limit. Make it expose X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset on every response, plus Retry-After on 429s. Then write a client that loops 20 requests, expects to hit 429 around request 6, reads Retry-After, sleeps, retries until all 20 succeed. Bonus: replace the in-memory counter with Redis and verify two parallel client instances share the same limit (without Redis, each instance would have its own 5-per-minute, doubling the actual capacity).
Hint
slowapi handles the FastAPI side; for the headers, you can compute them in a custom middleware that runs after the rate-limit check. Client: catch 429, read int(resp.headers['Retry-After']), time.sleep(), loop. The Redis bonus is the production-shape test — without distributed counters, scaling out your service multiplies the effective rate limit by replica count, which is a subtle production gotcha that bites teams the first time they horizontally scale.
Progress
Progress is local-only — sign in to sync across devices.