Content Negotiation — Agreeing on Format Without Pre-arranging

"You can serve JSON to one client, HTML to another, and a CSV download to a third — all at the same URL. The client tells you what they want via Accept; you tell them what you sent via Content-Type. The protocol handles the rest."

Two Headers and a Conversation

Content negotiation is the mechanism that lets the same resource speak multiple formats. Clients announce what they can handle; servers pick the best match. Two headers do the heavy lifting:

Accept (request) — "I can handle these formats, in this order of preference."
Content-Type (response) — "Here's what I actually sent."

Three more headers handle related axes:

Accept-Encoding / Content-Encoding — compression (gzip, br, zstd).
Accept-Language / Content-Language — locale (en-US, ko-KR).
Accept-Charset — character set. Largely vestigial now; UTF-8 won.

q-Values: How Clients Express Preference

An Accept header isn't just a list — it's a weighted list. Each option can carry a q= parameter from 0.0 ("don't send") to 1.0 ("want most"). When absent, q=1.0 is implicit.

Accept: application/json, text/html;q=0.9, */*;q=0.5

The client is saying: "Give me JSON if you can. If not, HTML is a 90%-okay fallback. Failing that, send anything (50% okay)." The server picks the highest-q option it actually supports. If nothing matches, the server returns 406 Not Acceptable.

The Vary Header — The Cache Coordination You Cannot Skip

If your server returns different responses for the same URL depending on a request header (Accept, Accept-Encoding, Authorization, Cookie), you MUST tell caches via the Vary response header. Otherwise the first response (say, JSON for a logged-in user) gets cached and served to subsequent requests that expected HTML (or no auth).

Vary: Accept, Accept-Encoding, Authorization

This tells the CDN: "This response is keyed by URL plus these three request headers. Don't serve this cached entry to a request whose Accept header differs." Forgetting Vary is the #1 source of "the wrong user saw the previous user's data" CDN bugs.

Content negotiation lets the protocol absorb format diversity. Without it, you'd need separate URLs (/users.json vs /users.html), which couples representation to URI and breaks bookmarks when you add a new format. With it, one URI serves all clients, and the wire layer does the matchmaking.

Server-Driven vs Agent-Driven Negotiation

Server-driven (the normal case) — the server reads Accept and picks. Most REST APIs work this way.

Agent-driven — the server replies with 300 Multiple Choices and a list of available formats; the client picks. Rare in practice; clients hate making that extra round trip.

Reactive negotiation (Vary-based) — server returns one format and uses Vary so caches do the right thing. This is what most production systems actually do, blending server-driven with cache coordination.

cwkPippa's Negotiation Reality

cwkPippa's API endpoints accept and produce only application/json — it's a single-format API, so content negotiation is degenerate (the server ignores Accept and always returns JSON). The frontend's static assets do the full dance: index.html negotiates language via Accept-Language, the CDN serves gzip or brotli depending on Accept-Encoding, and the response carries Vary: Accept-Encoding so the CDN keeps separate cached entries per encoding. SSE responses use Content-Type: text/event-stream — the only place cwkPippa serves a non-JSON content type.

Code

Content negotiation in action — same URL, different formats·bash

# Same URL, different Accept headers → different responses
curl -H 'Accept: application/json' https://api.example.com/users/42
# {"id":42,"name":"Pippa"}

curl -H 'Accept: text/html' https://api.example.com/users/42
# <html><body><h1>Pippa</h1></body></html>

curl -H 'Accept: text/csv' https://api.example.com/users/42
# id,name\n42,Pippa

# Client preferring JSON but accepting HTML as fallback
curl -H 'Accept: application/json, text/html;q=0.9' https://api.example.com/users/42
# JSON wins (q=1.0 implicit vs q=0.9)

# Client asking for something the server can't produce
curl -i -H 'Accept: application/xml' https://api.example.com/users/42
# HTTP/1.1 406 Not Acceptable
# Content-Type: application/json
# {"error":"only application/json or text/html supported"}

Server-side: pick format, set Vary, return the right Content-Type·python

# FastAPI — pick the response format based on Accept
from fastapi import FastAPI, Request, Response, HTTPException
from fastapi.responses import JSONResponse, HTMLResponse, PlainTextResponse
import csv, io

app = FastAPI()

def pick_type(accept: str, supported: list[str]) -> str | None:
    """Naive parser; production: use a library like 'starlette-context' or 'mediatype'."""
    accepted = [t.strip().split(';')[0] for t in accept.split(',')]
    for option in supported:
        if option in accepted or '*/*' in accepted:
            return option
    return None

USER = {'id': 42, 'name': 'Pippa'}
SUPPORTED = ['application/json', 'text/html', 'text/csv']

@app.get('/users/{uid}')
async def read_user(uid: int, request: Request):
    accept = request.headers.get('accept', 'application/json')
    chosen = pick_type(accept, SUPPORTED)
    if chosen is None:
        raise HTTPException(406, detail=f'only {SUPPORTED} supported')

    headers = {'Vary': 'Accept'}  # MANDATORY — tell caches the response varies
    if chosen == 'application/json':
        return JSONResponse(USER, headers=headers)
    if chosen == 'text/html':
        return HTMLResponse(f'<html><body><h1>{USER["name"]}</h1></body></html>', headers=headers)
    if chosen == 'text/csv':
        buf = io.StringIO()
        csv.writer(buf).writerows([['id', 'name'], [USER['id'], USER['name']]])
        return PlainTextResponse(buf.getvalue(), media_type='text/csv', headers=headers)

Compression negotiation — handled transparently by most HTTP clients·python

# Compression negotiation — the most common content negotiation in practice
import httpx

# Client tells server what compressions it can decode
resp = httpx.get(
    'https://creativeworksofknowledge.com/',
    headers={'Accept-Encoding': 'gzip, br, zstd'},
)
# Server picks the best supported one and returns:
# Content-Encoding: br  (Brotli)
# Vary: Accept-Encoding
print(resp.headers.get('content-encoding'))  # e.g. 'br'

# httpx (and most clients) handle decompression transparently
# resp.text is already decompressed when you access it
print(resp.text[:100])

Exercise

Extend the FastAPI server from Lesson 2.2 with content negotiation: the GET /items/{id} endpoint should return JSON when Accept: application/json is sent, HTML when Accept: text/html is sent, and CSV when Accept: text/csv is sent. Return 406 if none of these match. Set the Vary header correctly. Test with three curl calls (one per Accept). Bonus: deliberately leave off the Vary header, deploy behind any CDN, and watch one user receive the wrong response. (Don't actually do this in production. Just appreciate why.)

Hint

Vary: Accept on every response. Without it, the CDN treats your three responses (JSON, HTML, CSV) as interchangeable cached entries — the first one stored wins for every subsequent request. With Vary: Accept, the CDN keys its cache by URL+Accept, storing three separate entries.

Content Negotiation — Agreeing on Format Without Pre-arranging

Two Headers and a Conversation

q-Values: How Clients Express Preference

The Vary Header — The Cache Coordination You Cannot Skip

Server-Driven vs Agent-Driven Negotiation

cwkPippa's Negotiation Reality

Code

External links

Exercise

Progress

Comments 0