Long Polling: A Better Hack

The trick: hold the response

Long polling keeps the request shape but flips the timing. Instead of the server replying instantly with "no news," it holds the connection open and only responds when it has something to say (or when its own timeout expires). The client sees what looks like an instantaneous push, then immediately sends the next request to keep the channel primed.

Why this won the 2010s

Long polling worked through every corporate firewall and proxy that hated anything other than HTTP. Gmail used it for years. Facebook chat used it. It is still the safety net Socket.IO falls back to when WebSocket is blocked.

Why it is still a hack

Each response closes the connection. The TCP handshake gets re-paid. Proxies kill idle connections after a minute or two, so you must tune the server-side timeout below the proxy's threshold. Each hanging request consumes a server worker. And it is one-directional: the server can push fast, but the client still uses separate requests to send.

Code

FastAPI long-poll endpoint·python

from fastapi import FastAPI
import asyncio

app = FastAPI()
queue: asyncio.Queue = asyncio.Queue()

@app.get('/api/messages')
async def long_poll():
    try:
        # Wait up to 25s for something — under most proxy idle limits.
        msg = await asyncio.wait_for(queue.get(), timeout=25.0)
        return {'messages': [msg]}
    except asyncio.TimeoutError:
        # Empty response triggers the client to reconnect immediately.
        return {'messages': []}

Browser-side long-poll loop·javascript

async function longPoll() {
  while (true) {
    try {
      const res = await fetch('/api/messages');
      const { messages } = await res.json();
      if (messages.length) renderMessages(messages);
    } catch (err) {
      console.warn('long poll failed, backing off', err);
      await new Promise(r => setTimeout(r, 2_000));
    }
  }
}

longPoll();

Exercise

Run the FastAPI long-poll endpoint above with uvicorn main:app. In one terminal, push a message into the queue with a /admin/push endpoint you write. In another, hit GET /api/messages and watch how the connection only returns once the queue gets a message. Now hit it without pushing — observe the 25-second wait, then the empty response.