Rollouts and Canaries

~12 min · production, rollouts

Level 0Apprentice

0 XP0/100 lessons0/14 achievements

0/120 XP to next level120 XP to go0% complete

Don't switch in one step

A new prompt or model goes live the way other production code does: behind a feature flag, canaried at low traffic, monitored, then ramped up. Atomic switches are how regressions ship.

Rollout pattern

Internal canary — your team and select power users.
Canary traffic — 1–5% of production. Watch the dashboards.
Ramp — 10% → 25% → 50% → 100% over hours or days, depending on stakes.
Hold — at 100%, leave the previous version reachable for instant rollback.
Cleanup — once stable for ≥1 week, retire the previous version.

What you watch during ramp

Quality dashboards (eval flags, complaint rate).
Cost spikes per call.
Latency p95.
Refusal rate.
Specific failure modes you flagged in pre-ship review.

Code

Feature-flagged rollout·python

if growthbook.feature_value("prompt_v9", user_id, default=False):
    prompt = load("support.draft@9")
else:
    prompt = load("support.draft@8")

resp = call_model(prompt, request)

External links

Exercise

Set up a feature-flagged rollout for one prompt change. Define the ramp schedule. Run it. At each ramp step, decide go/no-go from the dashboards.

Progress

Progress is local-only — sign in to sync across devices.

← PreviousRate Limiting and Backpressure Next →Cost Engineering — Where the Money Goes

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.