Provider Differences Without Provider Worship

~26 min · openai, anthropic, gemini, portability

Level 0Window Watcher

0 XP0/50 lessons0/13 achievements

0/100 XP to next level100 XP to go0% complete

Same idea, different knobs

OpenAI prompt caching is automatic for eligible prompts and exposes cached_tokens in usage. Anthropic uses explicit cache_control breakpoints and reports cache_creation_input_tokens and cache_read_input_tokens. Gemini offers implicit and explicit context caching with its own token thresholds and TTL behavior.

Portable idea, provider-specific implementation

The portable idea is stable prefix reuse. The provider-specific implementation changes. Design your app around the idea, then adapt the adapter layer to the provider. Apps that hardcode one provider's caching mechanism become very awkward when adding a second.

Keep abstractions honest

Do not hide all provider details. Your telemetry should still record which provider, model, cache mode, and cached token count were used. The differences exist; pretending they do not creates the worst kind of bug — works on provider A, fails silently on B.

Code

Provider-neutral cache telemetry·yaml

cache_report:
  provider: "openai | anthropic | gemini"
  model: "..."
  cache_mode: "automatic | breakpoint | explicit"
  fresh_input_tokens: 0
  cached_input_tokens: 0
  ttl_seconds: null
  prefix_version: "rules-v3"

Adapter layer sketch·python

class CacheAdapter:
    def attach_cache_breakpoint(self, content_block): ...
    def parse_cache_usage(self, response) -> dict: ...

class AnthropicAdapter(CacheAdapter):
    def attach_cache_breakpoint(self, block):
        block["cache_control"] = {"type": "ephemeral"}
        return block
    def parse_cache_usage(self, response):
        u = response.usage
        return {"cached": u.cache_read_input_tokens,
                "created": u.cache_creation_input_tokens}

External links

Exercise

Design a cache telemetry object that could support OpenAI, Anthropic, and Gemini without pretending they are identical.

Hint

Include provider-specific fields as optional details, not as the core abstraction.