Explainable by Construction, Not by Explanation

Two Ways to Be Explainable

You can build a ranker and then add explanations to it, or you can build a ranker whose structure is the explanation. These sound similar and are not. The first produces a plausible story generated alongside the score — and a story that is generated separately can drift from the thing it describes, which is worse than no story at all. The second constrains the score to be a sum of named terms, so the explanation is not produced, it is read off.

For a personal reader the second is not a sacrifice. The interesting question is never "could a bigger model rank these better?" — it is "why is this on my shelf, and how do I make it stop?" A sum of named contributions answers both. A learned reranker answers neither, and buys accuracy nobody asked for on a corpus of a few hundred visible cards.

What the Structure Looks Like

Features are strings with namespaces — the publisher, the source that delivered it, and a bounded set of keyword tokens from the title and summary. The profile maps those strings to weights. An article's score is the sum, over its own features, of the profile weight for each, plus a freshness term. That is the whole model, and each addend is nameable.

Because it is a sum of named terms, the top few contributions are the explanation. The card can say it matched a subject you have been saving, or that it came from a publisher you keep opening, and that sentence is not a narrative about the score — it is the largest terms in it.

One caveat, and it is this lesson's own warning turned on itself: in the shipped scorer the freshness term is added to the total but is not among the contributions the card reports. So a card with any affinity match at all is ranked partly by recency and never says so — only a card with no contributions whatsoever falls back to reporting bare freshness. The structure makes drift impossible only for the terms it actually collects, and a term added outside that list is exactly the gap the approach is supposed to close. Collect every addend, or the guarantee is a habit rather than a property.

Cap the Match Count, and Notice What the Cap Implies

One detail that looks like tuning and is actually structural: an article may bank only a limited number of keyword matches. Without a cap, a long article about a familiar subject accumulates matches until it dominates purely by surface area, which rewards verbosity rather than relevance.

The cap has a consequence worth holding on to, because a later lesson turns on it: if an article gets six slots, then which six matters enormously. Six slots spent on grammatical particles are six not spent on real subjects, and no amount of correct arithmetic downstream can recover them.

Say It in the Payload

The explanation has to travel with the article, computed at scoring time, or it will be reconstructed later by different code and be subtly wrong. Ship the top contributions as data on the row. Then the interface can render them, a debugging view can dump them, and — the part that matters most — the reader can disagree with a specific term rather than with the shelf as a whole.

An explanation generated beside a score can be wrong; an explanation read out of a score cannot. Constrain the model so that its top terms are literally the reason, and explainability stops being a feature you maintain and becomes a property you cannot lose.

Code

A score as a sum of named terms, with `why` read off rather than written·python

EVENT_WEIGHTS = {
    "saved_at":    4.0,   # filing it is the strongest positive act
    "liked_at":    3.0,
    "shared_at":   3.0,
    "read_at":     1.0,   # opening is weak evidence, but it is evidence
    "disliked_at": -4.0,  # an explicit verdict, weighted like a save
}
FRESHNESS_WEIGHT = 1.5
KW_SCALE = 0.5
KW_MATCH_CAP = 6          # surface area must not beat relevance


def features(article: dict) -> dict[str, float]:
    """Namespaced strings. Every score term will be nameable because
    every feature already has a name."""
    feats = {f"outlet:{article['outlet']}": 1.0,
             f"source:{article['source_id']}": 1.0}
    for tok in tokens(article):
        feats[f"kw:{tok}"] = KW_SCALE
    return feats


def score(article: dict, profile: dict[str, float], now) -> tuple[float, list]:
    """Returns (score, why). `why` is not a story ABOUT the score -- it
    is the largest terms IN it, so the two cannot drift apart."""
    total, terms, matches = 0.0, [], 0
    for key, scale in features(article).items():
        weight = profile.get(key)
        if not weight:
            continue
        if key.startswith("kw:"):
            if matches >= KW_MATCH_CAP:
                continue
            matches += 1
        contribution = weight * scale
        total += contribution
        terms.append((key, round(contribution, 2)))

    fresh = FRESHNESS_WEIGHT * decay(article["published_at"], now,
                                    FRESHNESS_HALF_LIFE_DAYS)
    total += fresh
    # NOTE, and this is the shipped behaviour rather than the ideal
    # one: freshness moves the total but is NOT appended to `terms`,
    # so a card ranked partly by recency never says so. Appending it
    # here is the one-line fix; leaving it out is how a "cannot
    # drift" guarantee quietly becomes a convention. The shipped
    # fallback is `why or "fresh"`, which covers the empty case
    # and hides the mixed one -- the harder half.

    terms.sort(key=lambda t: -abs(t[1]))
    return total, terms[:3]        # travels with the row, in the payload

Exercise

Take any ordering in a product you use daily and try to state, in one sentence, why the top item is on top. Then check whether the product could tell you. If it could not, work out whether that is because the model cannot be decomposed, or merely because nobody surfaced the decomposition — those are very different problems with very different costs to fix.

Hint

A useful probe is whether the interface offers a way to say 'less like this'. Products that can decompose a score usually offer per-reason feedback, because they have somewhere to put it. Products that cannot tend to offer only a single blunt negative control, which is a strong hint about what is behind it.

Two Ways to Be Explainable

What the Structure Looks Like

Cap the Match Count, and Notice What the Cap Implies

Say It in the Payload

Code

External links

Exercise

Progress

Comments 0