Substitute Before You Compare

Some Shared Files Cannot Be Identical

Most of the shared layer is byte-identical everywhere, which is what makes the comparison trivial. A few files cannot be. The clearest case is an app-shell service worker: the caching strategy, the lifecycle handling, the update flow — all identical by design. But the cache name has to be unique per app, or two apps served from the same origin fight over one cache, and the list of shell URLs is different for every app because their entry points differ.

So those files ship as templates. The kit source contains placeholder tokens; the manifest carries a variable block per consumer; and the deploy substitutes before writing.

The Ordering Trap

Here is the mistake that is easy to make and annoying to diagnose: substitute on deploy, and compare the raw source on check. The result is that every templated file reports drift in every consumer, permanently, because of course the deployed file differs from the unsubstituted source — that is the entire purpose of the template.

The fix is one line of ordering: render the source for that consumer, then compare. The check must ask "does this file match what a deploy would write here right now", not "does this file match the source". For non-templated files those two questions have the same answer, which is exactly why the mistake survives testing until the first template arrives.

A verifier must compute the expected value the same way the producer does, including every transformation. The moment a producer applies a step the verifier does not, the verifier is checking a value that never existed. Template substitution is the common case; generated code, formatting passes, and minification are the same shape. If the producer transforms, the verifier transforms — or the check is measuring the wrong thing on purpose.

Generation Is Just a Bigger Transform

The same machinery covers a case that looks different but is not. One kit source is a plain data file: a catalog of the assistant brains, their labels, their effort options, their route paths. It is not deployed as-is anywhere. Instead, three transforms render it into a Python module, a TypeScript module, and a Swift file, and each of those is deployed to the consumers that need that language.

The check handles it without any special case, because rendering is just another transformation applied before comparison. And this is where a real benefit shows up: hand-writing three language bindings of one table is a guaranteed drift source, since one of them will be updated and the others forgotten. Generating them from one file means the drift check now covers not only "nobody edited a copy" but also "all three languages agree" — one mechanism, two invariants.

Code

The pipeline, in the order that makes the check correct·python

def expected_for(entry: dict, repo: str) -> str:
    """What a deploy WOULD write into `repo` right now, body only.

    Every step the producer applies must appear here, in the same
    order, or the check compares against a value that never exists
    on disk anywhere.
    """
    raw = (KIT_ROOT / entry["source"]).read_text()

    # 1. generation: one data file -> a language binding
    #    (validated inside; a malformed catalog fails the DEPLOY,
    #     which is much better than failing at app startup)
    raw = transform(raw, entry.get("transform"))

    # 2. per-consumer template substitution
    raw = substitute(raw, (entry.get("vars") or {}).get(repo))

    return raw     # 3. the header is added AFTER this, and stripped
                   #    again before comparison - it is never part of
                   #    the compared body


# A manifest entry for a templated source. The vars block is what
# makes one file correct in several places:
#
# {
#   "source": "kit/assets/sw.template.js",
#   "targets": {
#     "app-a": "frontend/public/sw.js",
#     "app-b": "frontend/public/sw.js"
#   },
#   "vars": {
#     "app-a": {"__KIT_APP_SLUG__": "app-a",
#               "__KIT_SHELL_LIST__": "'/', '/journeys'"},
#     "app-b": {"__KIT_APP_SLUG__": "app-b",
#               "__KIT_SHELL_LIST__": "'/', '/today'"}
#   }
# }


def test_check_is_clean_right_after_deploy():
    """The property that catches the ordering trap immediately.
    If deploy and check disagree about transformations, this fails
    the first time a templated file enters the manifest."""
    assert run(check=False) == 0
    assert run(check=True) == 0

Exercise

Find a templated or generated file in your project and write the deploy-then-check assertion for it: regenerate, then verify, and require both to pass. If it fails, you have found a transformation the verifier does not know about. Write down what that transformation is — it is part of the file's specification and it currently lives only in the generator.

Hint

Formatting is the transformation people forget most often. A generator that emits code and then runs a formatter over it has two steps; a checker that only runs the generator will report drift on every file, forever, and somebody will 'fix' it by loosening the comparison.

Some Shared Files Cannot Be Identical

The Ordering Trap

Generation Is Just a Bigger Transform

Code

External links

Exercise

Progress

Comments 0