The Pictures Were Buried, Not Missing

Ask Where the Thing You Want Already Is

After establishing that the enrichment everyone wanted could not help, the question changed from "how do we produce pictures?" to "do we already have pictures we are not showing?" That is a different query, and it is the one that ended the investigation.

Broken down by source, the answer was stark. Feeds coming directly from a publisher delivered an image with nearly every entry — a dozen of them at essentially one hundred percent, and a wire service in the nineties. Sources built from an aggregator's search query delivered images with almost none. The pictures were not missing from the store. They were sitting in it, on rows that never reached a screen.

Why the Good Rows Lose a Purely Chronological Shelf

The mechanism is volume, and it is worth stating as a general property rather than as a quirk of news. A broad search-query source publishes hundreds of items a day; a specialist publication publishes a few dozen. On a shelf ordered strictly by time, the loud source owns essentially the entire first screen — not because its articles are better or newer in any meaningful sense, but because it emits more per hour.

One shelf showed the shape perfectly: a dozen imageless rows from a query source on screen, while dozens of picture-bearing articles from two specialist publications sat on the same shelf, unseen, a scroll away. Chronological order is not neutral. It is a ranking function that weights by publication rate, and nobody chose it.

The Fix Costs Nothing

Because the material already exists, the correction is a render-time projection rather than a fetch: inside a recency window, let picture-bearing articles fill up to a floor of the visible region. Zero requests. No change to what enters the store, no change to what the fetch rounds do.

Two constraints keep it honest. The recency window stops it from promoting stale pictures — a shelf full of old images is a worse failure than a shelf of fresh headlines. And it reports how many rows it moved, so the shelf can say what it did rather than silently reordering.

The Other Half: Fix the Inputs Too

A render-time fix can only redistribute what exists, so the measurement also produced a shopping list. Several shelves turned out to have no picture-bearing source at all — every one of their sources was a search query. No projection can rescue those, and the answer is editorial rather than technical: add real publisher feeds, each one measured with the system's own parser before being added rather than assumed to work.

Before building a producer, check whether you are already discarding the thing you plan to produce. "We do not have enough X" and "we have X and are not surfacing it" feel identical from the outside and have completely different fixes — one costs a subsystem, the other costs a sort.

Code

Promotion as a render-time projection: no requests, bounded, and self-reporting·python

HEAD_CARDS = 12                  # the visible region
IMAGE_FLOOR = 8                  # how many of it may carry pictures
IMAGE_FLOOR_WINDOW_HOURS = 24.0  # never promote a stale picture


def apply_image_floor(rows: list[dict], limit: int) -> list[dict]:
    """Let pictures reach the head, without letting stale ones in.

    Zero requests: the pictures already exist on rows a purely
    chronological shelf buries, because a broad query source emits
    hundreds of items a day and a specialist publication a few dozen.
    """
    screen = min(HEAD_CARDS, limit)      # the SCREEN, not the page
    head = rows[:screen]
    have = sum(1 for r in head if r.get("image_url"))
    if have >= IMAGE_FLOOR:
        return rows                      # already fine: promote nothing

    now = datetime.now(timezone.utc)
    fresh = [r for r in rows
             if r.get("image_url")
             and _age_hours(r, now) <= IMAGE_FLOOR_WINDOW_HOURS][:IMAGE_FLOOR]
    if len(fresh) <= have:
        return rows                      # nothing fresher to promote

    picked = {r["id"] for r in fresh}
    keep = list(fresh)
    for row in head:                     # backfill the rest of the screen
        if len(keep) >= screen:
            break
        if row["id"] not in picked:
            keep.append(row)
            picked.add(row["id"])

    keep.sort(key=lambda r: _age_hours(r, now))   # still newest-first
    was_on_screen = {r["id"] for r in head}
    for row in keep:
        if row["id"] not in was_on_screen:
            row["promoted"] = True       # say what you moved
    return keep + rows[screen:]

Exercise

Take a list in your system that is sorted by time and group its visible head by whatever produces the items. If one producer owns most of the head, compute how much that producer emits relative to the others. Then decide whether that share reflects an editorial judgment you would actually defend, or merely a difference in throughput nobody chose.

Hint

Do the grouping on the visible head specifically, not on the whole result set. Over a large window the shares often look reasonable, because the quiet producers eventually accumulate. The distortion is concentrated in exactly the region a person sees, which is the region no aggregate query is looking at.

Ask Where the Thing You Want Already Is

Why the Good Rows Lose a Purely Chronological Shelf

The Fix Costs Nothing

The Other Half: Fix the Inputs Too

Code

External links

Exercise

Progress

Comments 0