A Source Is Its Configuration, Not Its Name

The Bug That Reports Success

Here is a failure with no error message anywhere. A reader adds a second feed from a publisher it already carries — the world section, when it already had the top-stories section. The interface confirms the source was added. The shelf binding appears. Fetch rounds run clean. And the new shelf fills with the old feed's articles, forever, because the two rows were treated as the same source.

The cause is that the code deduplicated on the name. And the name, in this system, was not typed by a human at all — the client derives a label for a syndication source from its hostname, so two different paths on one host produce the same label. Get-or-create matched the label, found the existing row, returned it, and reported success. Nothing was wrong from any code path's point of view.

Identity Belongs to the Thing That Determines Behavior

The general principle is worth stating plainly, because it generalizes far past feeds: an entity's identity is whatever determines what it does. For a source, what it does is entirely decided by its kind and its configuration — which endpoint, which query, which locale. The name determines nothing; it is a label for humans, and labels are allowed to collide.

Deduplicating on a label rather than on the behavior-determining fields produces exactly this class of bug: a silent aliasing where the system believes it has two things and has one. Note that the reverse mistake — never deduplicating at all — is not harmless either. Identical configurations genuinely are one feed, and giving each shelf its own row means fetching the same endpoint several times per round, which is precisely the impoliteness the previous lesson was about.

Compare Parsed, Not Serialized

One implementation detail carries real weight. Configurations are usually stored as serialized JSON, and it is tempting to compare the stored strings. Do not. Two configurations differing only in key order — or in whitespace, or in unicode escaping — are the same configuration, but their serializations differ. Parse both sides and compare the resulting structures, so that identity is a property of the configuration rather than of how it happened to be written.

Then De-Collide the Label

Once identity moves to the configuration, the name is free to be merely a display string — and it must be allowed to collide gracefully. A new row whose label is taken gets a suffix. That is all. The label being ugly is a much smaller problem than the label being load-bearing.

Deduplicate on what determines behavior; disambiguate what is only displayed. When those two responsibilities land on the same field, one of them will quietly lose — and it will be the behavioral one, because display collisions are visible and behavioral collisions are not.

Code

Identity on the configuration, disambiguation on the label·python

def _get_or_create_source(con, name, kind, cfg, now):
    """A source IS its (kind, config); the name is only a label.

    Matching on the label instead meant two different feeds from one
    host collided -- the client derives a name from the hostname, so
    adding a publisher's World feed after its Top feed silently bound
    the Top feed and reported success.
    """
    # Identity: parsed comparison, so key order never decides it.
    for row in con.execute(
        "SELECT id, config FROM sources WHERE kind=?", (kind,)
    ):
        if json.loads(row["config"] or "{}") == cfg:
            return int(row["id"])        # genuinely the same feed

    # Display: the label is allowed to collide, so de-collide it.
    label, suffix = name, 2
    while con.execute(
        "SELECT 1 FROM sources WHERE name=?", (label,)
    ).fetchone():
        label = f"{name}-{suffix}"
        suffix += 1

    cur = con.execute(
        "INSERT INTO sources(kind, name, config, created_at) VALUES(?,?,?,?)",
        (kind, label, json.dumps(cfg, ensure_ascii=False), now),
    )
    return int(cur.lastrowid)

Exercise

Find a get-or-create in code you own and write down which column its lookup matches on. Then ask whether that column fully determines the entity's behavior. If it does not, construct the two inputs that would alias — same lookup key, different behavior — and check what the function returns. Decide whether the caller could possibly notice.

Hint

The aliasing pair usually exists already in your data; you just have not looked. Group by the lookup column and count distinct values of the behavior-determining columns within each group. Any group with more than one is a live alias, not a hypothetical.

The Bug That Reports Success

Identity Belongs to the Thing That Determines Behavior

Compare Parsed, Not Serialized

Then De-Collide the Label

Code

External links

Exercise

Progress

Comments 0