Load Balancing

~12 min · production, load-balancer, sticky-sessions

Level 0Poller

0 XP0/60 lessons0/10 achievements

0/120 XP to next level120 XP to go0% complete

The sticky-session problem

WebSocket connections are persistent and stateful — they live on a specific server process. When a client reconnects, they may hit a different server with no memory of their session, room membership, or auth context. Sticky sessions ensure that reconnections from the same client land on the same server.

Three sticky strategies

IP hash: simplest, works without cookies, breaks behind shared NATs (university wifi, corporate VPN). Cookie-based affinity: precise, but cookies are not always set in time for the first WebSocket upgrade. Connection-aware (least-conn): distributes load evenly but provides no affinity — only good when your servers are stateless or you have a Redis bridge to compensate.

Code

Nginx sticky via ip_hash·nginx

upstream websocket_servers {
    ip_hash;                # same client IP -> same backend
    server backend1:8000;
    server backend2:8000;
    server backend3:8000;
    keepalive 64;
}

server {
    listen 443 ssl http2;
    server_name api.example.com;

    location /ws {
        proxy_pass http://websocket_servers;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_read_timeout  3600s;
        proxy_send_timeout  3600s;
    }
}

HAProxy sticky via cookie·text

backend websocket
    cookie SRV insert indirect nocache
    server srv1 backend1:8000 check cookie srv1
    server srv2 backend2:8000 check cookie srv2
    server srv3 backend3:8000 check cookie srv3
    timeout tunnel 1h

External links

Nginx WebSocket proxying

Exercise

Set up two FastAPI WebSocket servers behind Nginx with ip_hash. Connect from two devices on the same NAT (your phone and laptop on home wifi); confirm they may land on the same backend. Now connect from two different IPs; confirm they spread across backends. Document what you saw.