Cost Optimization Routing

The single biggest lever

Pro is ~22.5× more expensive per output token than Flash-Lite. Most of your traffic doesn't need Pro. A smart router that sends routine work to Flash-Lite, default work to Flash, and only-when-required work to Pro typically cuts production bills 70–90%.

Routing inputs

The decision is based on:

Reasoning needed? Multi-step logic, code, math → Pro.
Context size? > 200K tokens → Pro (only Pro handles long context well; Flash and Flash-Lite hit quality drops).
Tool use? Complex multi-tool agentic loops → Pro. Simple single-tool calls → Flash.
Latency? Sub-500ms requirement → Flash-Lite (smallest, fastest TTFT).
Volume? High-volume simple tasks → Flash-Lite.

Real numbers

Strategy	Relative cost	When
Always Pro	1.0×	Maximum quality, no cost ceiling
Smart routing	~0.3×	Most production apps
Flash-Lite only	~0.04×	High-volume simple tasks
Caching enabled	~0.1× per cached call	Repeated context (PDF Q&A)
Batch API (offline)	0.5×	Async pipelines

Code

Smart router — minimal version·python

class SmartRouter:
    """Pick the cheapest capable Gemini model for a request."""

    def route(
        self,
        needs_tools: bool = False,
        needs_reasoning: bool = False,
        max_context: int = 0,
        latency_budget_ms: int = 10_000,
    ) -> str:
        # Tight latency budget — go straight to Flash-Lite
        if latency_budget_ms < 500:
            return 'gemini-2.5-flash-lite'

        # Reasoning or huge context — Pro is the only safe choice
        if needs_reasoning or max_context > 200_000:
            return 'gemini-2.5-pro'

        # Tools without reasoning — Flash handles fine
        if needs_tools or max_context > 50_000:
            return 'gemini-2.5-flash'

        # Simple high-volume — cheapest model
        return 'gemini-2.5-flash-lite'

router = SmartRouter()
model = router.route(
    needs_tools=True,
    needs_reasoning=False,
    max_context=80_000,
)
print(model)  # 'gemini-2.5-flash'

Inspect-then-pick — let the prompt classify itself·python

async def route_by_classification(prompt: str) -> str:
    """Use Flash-Lite to decide which model the real prompt deserves."""
    classifier = await client.aio.models.generate_content(
        model='gemini-2.5-flash-lite',
        contents=(
            'Classify the following user request into exactly one bucket: '
            'CHAT (simple Q&A), CODE (writing or debugging code), '
            'REASON (multi-step logic), or AGENT (multi-tool workflow). '
            f'Output only the bucket name.\n\nRequest: {prompt}'
        ),
        config={'max_output_tokens': 10, 'temperature': 0.0},
    )
    bucket = classifier.text.strip().upper()
    return {
        'CHAT':   'gemini-2.5-flash-lite',
        'CODE':   'gemini-2.5-flash',
        'REASON': 'gemini-2.5-pro',
        'AGENT':  'gemini-2.5-pro',
    }.get(bucket, 'gemini-2.5-flash')

Exercise

Build the smart router from the first code block. Wire it into the GeminiAdapter from the previous lesson so the model is chosen per-call instead of fixed at construction. Run 100 mixed prompts (some chat, some reasoning, some long context) and log which model was chosen. Compute the cost vs always-Pro and always-Flash-Lite baselines.