Reserve, Then Round-Robin: Degrading Depth Instead of Breadth

Two Classes of Section, Two Policies

Once you accept that a corpus is an allocation, the sections sort themselves into two kinds. Some are small, unique and irreplaceable: a handful of temperature readings, two articles read in full, yesterday's summary. Others are bulky, repetitive and elastic: headlines, of which there are always more and each of which adds a little.

These deserve opposite policies. The small unique sections should be reserved in full, because a fraction of them is nearly worthless — half a temperature reading tells you nothing. The bulky elastic section should take whatever remains and stop cleanly, because it degrades gracefully by construction: fewer headlines is a smaller survey, not a broken one.

Within the Elastic Section, Order Matters Too

Having reserved the fixed sections, you still have to decide how the remaining allowance is spent among many shelves. The natural implementation walks shelf by shelf, and it reproduces the original bug one level down: the first few shelves are fully represented and the last several are entirely absent.

Round-robin by rank fixes it. Take every shelf's top headline, then every shelf's second, and so on. Running out of budget then costs each topic a little depth rather than erasing whole topics — and for a survey of the day, a missing subject is a far bigger defect than slightly thinner coverage of each one.

Continuity for Free

One reserved section deserves special mention because it is a spending decision disguised as a feature. "What changed since yesterday" is a genuinely useful thing for a daily brief to say, and the obvious implementation is a second generation that diffs two days. That would double a standing cost.

It is also unnecessary, because it is the same question this generation is already answering. Including yesterday's brief as a reserved section costs a few hundred characters and no additional call. The model can then say what has moved — and, usefully, what it got wrong yesterday. Before adding a generation step, check whether the answer is already reachable inside the one you are about to make.

Protect the Reservation With a Test That Floods

A reservation is only real if something enforces it. The test that matters does not check a normal day; it deliberately floods the elastic section with far more content than the budget allows, and then asserts the reserved sections are still present and intact. That test fails on the original implementation and passes on the budgeted one, which is exactly what a regression test for this class of bug should do.

Reserve what cannot degrade; ration what can. The question to ask of every section is not "how important is this?" but "is half of it worth anything?" — and that question sorts sections into the two policies far more reliably than importance ever does.

Code

Round-robin over the elastic allowance, and the flooding test that protects the reservation·python

def _fill_round_robin(shelves: list[list[str]], budget: int):
    """Spend the elastic allowance across shelves by RANK.

    Shelf-by-shelf would reproduce tail truncation one level down: the
    first shelves fully represented, the last ones absent. By rank,
    running out costs each topic a little depth instead of erasing
    whole topics -- and a missing subject damages a survey far more
    than thinner coverage of every subject.
    """
    lines, spent, dropped = [], 0, 0
    deepest = max((len(s) for s in shelves), default=0)
    for rank in range(deepest):
        for shelf in shelves:
            if rank >= len(shelf):
                continue
            cost = len(shelf[rank]) + 1
            if spent + cost > budget:
                dropped += 1
                continue          # keep going: a later shelf may fit
            lines.append(shelf[rank])
            spent += cost
    return lines, dropped


# The test that makes the reservation real. It does NOT test a normal
# day -- a normal day fits, and passes either way. It floods.
def test_a_wide_day_cannot_evict_the_reserved_sections(con):
    seed_shelves(con, count=40, headlines_each=200)   # far over budget
    corpus, stats = build_corpus(con)

    assert "X temperature" in corpus, "reserved section was evicted"
    assert "Yesterday's brief" in corpus
    assert stats["samples"] > 0
    assert stats["headlines_dropped"] > 0     # the elastic part absorbed it
    assert len(corpus) <= CORPUS_MAX_CHARS

Exercise

Take a bounded output you assemble — a prompt, a digest, a summary payload — and sort its sections by asking of each: is half of this worth anything? Then check whether your current implementation gives the reserve-class sections priority. Write the flooding test even if the answer is yes, because the ordering will drift the next time someone adds a section.

Hint

New sections get appended at the end, which in a truncating assembly means every addition silently becomes the lowest priority regardless of how important it is. The flooding test is what makes that drift fail loudly instead of quietly changing what your system sends.

Two Classes of Section, Two Policies

Within the Elastic Section, Order Matters Too

Continuity for Free

Protect the Reservation With a Test That Floods

Code

External links

Exercise

Progress

Comments 0