Original-First: The Link Out Is the Primary Act

Two Copies, One Original

The moment a reader caches a summary, extracts an article body, or renders text in its own typography, it is holding a working copy of something it does not own. That is fine — it is what reading tools do — but only if the relationship stays explicit. The canonical original is primary; everything the reader holds is a projection for convenience.

Stated as a rule: an article row always carries the original URL, that URL is always reachable in one deliberate action, and extraction never becomes the destination. In-app reading can be the default click — convenience is not a sin — as long as the way out is one tap and extraction failure degrades to the source's own summary rather than to something invented.

Canonicalization Is Identity, Not Tidiness

URLs arrive decorated. Campaign parameters, click identifiers from whichever platform referred them, per-newsletter tags — none of which name the article, all of which change the string. If you hash the raw URL to decide whether you have seen an article before, you will store the same piece three times because it was linked from three places.

So canonicalization has a job beyond hygiene: it produces the identity. Lowercase the scheme and host, drop the fragment, and strip the parameters that are known to be tracking — anything with a campaign prefix, plus the specific click identifiers the big platforms attach. Keep everything else, because a query parameter you do not recognize may well be the article id. Then hash the result. That hash is what a uniqueness constraint should be on.

Strip on the Way Out, Too

The same canonical form is what you hand to anyone else. When the reader shares an article, the link it publishes is the cleaned canonical URL — not the decorated one it happened to receive, and not a shortener. A shortener would make the reader an intermediary in someone else's click path and hide the destination from whoever receives it; passing along the tracking parameters would forward surveillance the reader has no reason to forward.

Where the Rule Bites

Original-first is easy to agree with and easy to erode, because each erosion is a small convenience. Caching the extract forever is convenient. Rendering only your version is convenient. Letting the share sheet default to your own reader page instead of the publisher's is convenient. The discipline is not refusing all of these; it is noticing that each one moves the primary act, and keeping the original link one unambiguous tap away no matter how good the copy gets.

A working copy is legitimate exactly as long as it points back. The test is not whether you cache — every reader caches — but whether a person holding your rendering can reach the publisher's in one action, and whether the link you hand to a third party is the publisher's, cleaned, and nobody else's.

Code

Canonicalization as identity — strip tracking, keep the unknown·python

_TRACKING_PREFIXES = ("utm_",)
_TRACKING_KEYS = {
    "fbclid", "gclid", "igshid", "mc_cid", "mc_eid",
    "cmpid", "ocid", "smid", "sref",
}


def canonical_url(url: str) -> str:
    """The canonical form is BOTH the identity we dedupe on and the link
    we hand to anyone else. Strip what is known to be tracking; keep
    everything else, because an unrecognized parameter may be the
    article id and dropping it would break the link."""
    url = (url or "").strip()
    if not url:
        return ""
    parts = urlsplit(url)
    query = [
        (k, v)
        for k, v in parse_qsl(parts.query, keep_blank_values=True)
        if not k.lower().startswith(_TRACKING_PREFIXES)
        and k.lower() not in _TRACKING_KEYS
    ]
    return urlunsplit((
        parts.scheme.lower(),
        parts.netloc.lower(),
        parts.path,
        urlencode(query),
        "",              # the fragment never identifies an article
    ))


def url_hash(url: str) -> str:
    """What the UNIQUE constraint is on. Hash the canonical form, never
    the raw string: the same article linked from three places arrives
    as three different strings and one identity."""
    return hashlib.sha256(canonical_url(url).encode("utf-8")).hexdigest()[:16]

Exercise

Collect a dozen links to the same handful of articles from different places — a newsletter, a social post, a search result, a messaging app. Canonicalize them by hand with a tracking denylist and count how many distinct identities you end up with versus how many distinct articles there really were. Then find one parameter you were unsure about, and check whether removing it still resolves.

Hint

The interesting failures are not the campaign parameters, which are obvious. They are the per-platform click identifiers you have never seen, and the one site that puts something load-bearing in the query string right next to them. That is exactly the pair a naive allowlist gets backwards.

Two Copies, One Original

Canonicalization Is Identity, Not Tidiness

Strip on the Way Out, Too

Where the Rule Bites

Code

External links

Exercise

Progress

Comments 0