Cost at Scale

Per-connection resource cost

Each WebSocket connection consumes: one file descriptor, ~10-50KB of memory (read/write buffers + framework metadata + your per-connection state), zero CPU at idle, and CPU proportional to message rate when active. 100K connections fit on one beefy box. 1M is achievable but takes care. 10M needs sharding by region or a managed service.

OS limits matter

Linux defaults to 1024 file descriptors per process. Without raising this, you cannot open 1024 WebSocket connections on one server. Raise via ulimit, limits.conf, and sysctl. Tune net.core.somaxconn and tcp_max_syn_backlog to handle reconnect storms after a network blip.

Self-host vs managed: the honest tradeoff

Self-host is cheap at 1K connections, expensive at 10M. Managed is expensive at 1K connections, cheap at 10M because you get scale and on-call expertise without hiring a team. The crossover for most apps is around 100K-1M concurrent. cwkPippa is firmly in self-host territory; a global SaaS chat product probably is not.

Code

Tuning Linux for high connection counts·shell

# /etc/security/limits.conf
*  soft  nofile  1000000
*  hard  nofile  1000000

# /etc/sysctl.conf
net.core.somaxconn         = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.tcp_fin_timeout   = 15
net.core.netdev_max_backlog = 5000

# Reload
sudo sysctl -p
ulimit -n 1000000

Decision matrix at scale·text

| Scale                      | Recommendation                          |
| -------------------------- | --------------------------------------- |
| < 1K concurrent            | Single FastAPI server, raw WebSocket    |
| 1K - 100K concurrent       | 2-5 servers + Redis Pub/Sub             |
| 100K - 1M concurrent       | Sharded servers + Redis cluster + LB    |
| 1M+ concurrent             | Strongly consider Ably / Pusher / etc.  |
| Tight team or deadline     | Managed service from the start          |
| Custom protocol or trading | Self-host at any scale                  |

Exercise

Pick three real services from your daily life — a chat app, a stock app, a notification app. Estimate their concurrent WebSocket connections (look up DAU and assume 30% peak online). Place each on the decision matrix. Argue out loud why each company would (or would not) pick the recommendation in the matrix, given what you know of their team and product.