One Standing Spend, Named in the Invariants

Where Background Spend Comes From

Nobody decides to burn tokens on articles nobody reads. It arrives one reasonable feature at a time. Pre-summarize everything so opening is instant. Pre-translate, since the reader sometimes wants it. Score each article with a model for better ranking. Each proposal is defensible in isolation, each multiplies a per-article cost by a daily arrival rate in the thousands, and the total appears at the end of a billing period rather than in any review.

The structural defense is not a budget alarm — an alarm tells you afterwards. It is a rule stated as an invariant: metered work happens when a person asks, with a fixed and enumerated set of exceptions.

Enumerate the Exception, Do Not Discover It

A daily briefing is worth one scheduled generation. That is a real exception and the rule has to admit it. The important part is that it is written down as the exception — one item, named, with the reason attached — rather than being the first of an unbounded set.

An enumerated exception gives you the question to ask of every future proposal: does this add a second standing spend? That question is answerable in a design discussion, which is where the decision should be made. Without it, each addition is judged on its own merits, and every individually-reasonable addition passes.

Bound the Scheduled One Too

Even the sanctioned spend needs a guard, because a scheduled job and a manual trigger can both fire. The manual refresh is a person asking, so it is allowed by the rule — but the two can overlap and produce two generations for one day.

A lock is the answer, and its behavior on contention is the design decision: it should decline rather than queue. Queuing turns a double-trigger into two sequential spends slightly later, which is the outcome the guard exists to prevent. Declining and returning what is already being written toward is what keeps one trigger from becoming two charges. Be careful about what that buys, though: it stops two briefs being written at once, not a second brief later the same day. A manual refresh is a person asking, so the rule permits it — the spend that is standing, and singular, is the scheduled one.

Attribute Every Token to a Tap

The practical form of the rule is an attribution test you can apply to any spend: point at the human action that caused it. Opening an article, pressing a queue button, tapping translate, asking a question — each of those is a tap, and a tap is authorization. The daily brief points at the invariant instead, which is why the invariant has to name it.

If a spend can point at neither, it is background burn no matter how useful the output is. That is the whole test, and it takes ten seconds to apply.

Enumerate standing costs; never let one arrive as a side effect. A rule with one named exception is enforceable, because the next proposal has to argue for becoming the second. A rule with an unwritten exception has none, because nobody can tell which addition crossed the line.

Code

The single scheduled spend, guarded by a lock that declines rather than queues·python

_brief_lock = threading.Lock()      # a LOCK, not a queue


async def generate_brief(kind: str = "manual") -> dict:
    """The scheduled spend, plus the manual refreshes a person asks for.

    Both the scheduler and a tapped refresh reach this, and they live
    in different event loops -- hence a threading lock rather than an
    asyncio one, which would protect nothing across loops.
    """
    # DECLINE on contention, never queue: queuing turns a double
    # trigger into two sequential spends slightly later, which is
    # exactly what this guard exists to prevent. Hand back what is
    # already being written toward instead.
    if not _brief_lock.acquire(blocking=False):
        return {**_latest_brief(), "already_running": True}
    try:
        return await _generate_brief(kind)
    finally:
        _brief_lock.release()


# Be precise about what this guarantees. It prevents two briefs being
# written AT ONCE. It does NOT prevent a second brief today -- a
# manual refresh is a person asking, and the rule allows that by
# design. The standing spend is the SCHEDULED lane, and that one is
# singular. Conflating 'one a day' with 'one at a time' would be
# claiming a guard the code does not implement.

Exercise

List every place your system spends metered money — model calls, paid APIs, compute jobs — and for each, name the human action that authorizes it. Anything that cannot point at one is a standing cost. Then check whether your standing costs are written down anywhere a design review would encounter them.

Hint

The ones that resist attribution are usually described with words like 'pre', 'warm', 'background' or 'sync'. Those prefixes mark work done in anticipation of a request, which means the request may never come — and if the output is not reused by a later tap, the anticipation was the entire cost.

Where Background Spend Comes From

Enumerate the Exception, Do Not Discover It

Bound the Scheduled One Too

Attribute Every Token to a Tap

Code

External links

Exercise

Progress

Comments 0