The Queue Is Explicit Taps, Deferred

Waiting Is the Problem, Not Processing

Pre-processing an article makes it open instantly, and doing it on demand makes the reader wait. The naive resolution is to process everything in advance, which is background burn on a firehose. The better one comes from watching how someone actually reads: they browse headlines first, then read. Those are separate sessions minutes apart.

So let the browsing pass mark what will be read. A queue button on the card is a tap that costs nothing at the time and authorizes work to happen before the reading pass begins. Nobody waits, nothing is spent on articles that were never chosen.

Why This Is Not an Exception to the Rule

A background worker that spends money looks like exactly what the previous lesson forbade. The distinction is who decided. Every item in the queue was put there by an explicit tap; the worker defers requested work to a convenient moment. A worker that scanned the shelf and picked promising articles itself would be the violation — same code, same cost, entirely different relationship to the person paying.

The property to preserve is that the queue is only ever populated by human action. The moment anything else can add to it, the worker becomes an autonomous spender and the rule is gone.

Two Lanes, One Claim

Two paths can now process the same article: the worker, and a direct open by someone who did not wait. They must not both pay.

The natural instinct is an in-process lock, and it does not work here — the request handler and the worker generally live in different event loops or threads, and an asyncio lock binds to the loop that first uses it, so the other side cannot wait on it. The claim has to live where both can see it, which means the database, with a timestamp so a crashed holder cannot block the article forever. The loser of the race does not fail; it polls briefly for the winner's cached result.

Finish What Was Started

One more failure mode, found in live use: a person opens an article, the processing turn begins, they close it two seconds later, and the request is cancelled — taking the turn with it. The money was spent and nothing was cached. Measured on a real day: twenty extractions, six cached.

Caching is the server's responsibility, not a side effect of the client staying connected. Running the turn shielded from cancellation means a closed reader loses the view, not the work. If you have already paid for a result, finish it and store it — the caller's interest and the value of the artifact are unrelated.

Deferring requested work is scheduling; choosing work yourself is spending. The same worker is either one depending on how its queue is filled, so the property worth enforcing is not what the worker does but who is allowed to put things in front of it.

Code

A queue fed only by taps, a cross-loop claim, and a turn that survives the client·python

QUEUE_WORKER_INTERVAL_SECONDS = 20


async def queue_worker_tick(con):
    """Serial, oldest-first. Every item here was an explicit tap, so
    this defers requested work -- it never chooses work itself. If
    anything but a human action could enqueue, this would become an
    autonomous spender."""
    article = store.oldest_queued_uncleaned(con)
    if not article:
        return
    if not store.claim_clean(con, article["id"]):
        return                      # the direct lane got there first
    try:
        await clean_article(con, article)
    finally:
        store.release_clean_claim(con, article["id"])


async def open_article_reader(con, article_id: int):
    """The direct lane. Cross-lane 'don't pay twice' is a DB claim with
    a TTL, not an asyncio lock: an asyncio lock binds to the loop that
    first uses it, so the worker's loop cannot wait on it at all."""
    if cached := store.clean_text(con, article_id):
        return cached

    if store.claim_clean(con, article_id):
        # Shielded: closing the reader mid-turn must not abort caching.
        # Measured before this existed -- 20 extractions, 6 cached.
        # The money is spent either way; the artifact is what is at risk.
        task = asyncio.create_task(clean_article(con, article_id))
        _shielded.add(task)                     # keep a strong reference
        task.add_done_callback(_shielded.discard)
        return await asyncio.shield(task)

    return await _poll_for_winners_result(con, article_id)

Exercise

Find an expensive operation in your system triggered by a user request, and check what happens if the client disconnects halfway. Does the work stop, and if so, was anything already paid for? Then check whether a second path — a retry, a worker, a sibling request — could start the same work concurrently, and what stops both from completing.

Hint

Most frameworks cancel the handler on disconnect, which is correct for cheap idempotent reads and wrong for anything that spends. The tell is a paid operation awaited directly in a route function with no shielding and no cache write before the await returns.

Waiting Is the Problem, Not Processing

Why This Is Not an Exception to the Rule

Two Lanes, One Claim

Finish What Was Started

Code

External links

Exercise

Progress

Comments 0