Picture the moment the whole mechanism exists to survive. Someone is deep in a consumer repository, chasing a bug, and the fix is one line inside a vendored file sitting right there in the editor. The header says do not edit here. They have already stopped reading headers, because they are three hours into a bug.
A check that runs only in the shared repository reports this later, to somebody else, about a repository the editor was not looking at. A check that runs in the consumer's own test suite goes red for them, in the repository they are already in, with a message naming the file to edit instead. Same check, same script, completely different outcome — because the correction arrives while the context is still loaded.
So each consumer runs the drift check from its own suite. It shells out to the shared repository's script in check mode, scoped to that consumer, and asserts a zero exit.
The Rule Was Written as a Path, and That Was the Bug
The convention had been recorded like this: every consumer has a test file at a particular path under its backend directory. It worked for every consumer that had a backend directory.
Then one joined that is frontend-only. It vendored shared files for a week with no check at all. Nothing failed, because a test that does not exist cannot fail — and there was no inventory anywhere asking whether each consumer's check was present. The repository quietly complied with nothing while looking exactly like a compliant member of the family.
The correction was not to give that repository a backend. It was to notice that the rule's real content had been mis-stated. The requirement is every consumer enforces the check; the path was only ever an implementation detail of how the majority happened to do it. The frontend-only consumer ran the same check from its own JavaScript test runner. Same assertion, different host.
State the requirement, not the implementation — and be most suspicious when the implementation is a file path. A rule expressed as a location silently exempts anyone whose structure cannot host that location, and the exemption is invisible because it manifests as an absence. When a new participant joins, the right question is never "can you host this file" but "which of your existing mechanisms will enforce this".
Verify That the Check Can Fail
When the frontend-only consumer's check was added, it was not merely observed to pass. A vendored file was deliberately edited, the suite was run, and the failure was confirmed — then the edit was reverted.
That step is easy to skip and it is the only thing separating a real check from a decorative one. A green test proves nothing about a check's sensitivity: a check with an inverted condition, a wrong path, a swallowed exception, or a filter that matches nothing all pass beautifully forever. The first thing a new guard should do is fail on purpose.
Code
The same assertion, hosted by whichever suite the consumer has·python
# --- Consumer with a Python suite --------------------------------
import subprocess, sys
from pathlib import Path
import pytest
CHECKOUT = Path(__file__).resolve().parents[2]
KIT_ROOT = CHECKOUT.parent / "the-kit" # beside the MAIN checkout;
# from a worktree, ask git
SYNC = KIT_ROOT / "sync" / "kit_sync.py"
def on_the_fleet() -> bool:
"""A fleet machine always carries the kit beside the repo, so an
absent kit THERE is a defect, not a reason to skip."""
label = Path.home() / ".config" / "fleet-label"
return label.is_file() and bool(label.read_text().strip())
def test_vendored_files_match_the_kit() -> None:
if not SYNC.is_file():
if on_the_fleet():
pytest.fail(f"fleet machine, no kit at {KIT_ROOT}: clone it there")
pytest.skip(f"off the fleet and no kit at {KIT_ROOT}: not checked here")
result = subprocess.run(
[sys.executable, str(SYNC), "--check", "--repo", "this-app",
"--repo-root", str(CHECKOUT)], # judge THIS checkout's copies
capture_output=True, text=True, timeout=30,
)
assert result.returncode == 0, (
"drift detected - edit the kit, not the vendored copy:\n"
+ result.stdout + result.stderr
)
# --- Consumer with only a JS suite -------------------------------
# Same script, same flags, same assertion. The rule is "a suite runs
# this", and it never cared which language that suite is written in.
# (This is how the frontend-only consumer hosted it until it became a
# native app; it now calls the package's drift helper from Swift.)
#
# import { execFileSync } from "node:child_process";
# import { existsSync } from "node:fs";
# import { test, expect } from "vitest";
#
# const SYNC = "../the-kit/sync/kit_sync.py";
#
# test.skipIf(!existsSync(SYNC))("vendored files match", () => {
# expect(() =>
# execFileSync("python3", [SYNC, "--check", "--repo", "this-app"])
# ).not.toThrow();
# });
# --- The inventory that would have caught the gap ------------------
# One assertion in the KIT's own suite: every repo that appears as a
# deploy target must also appear in the enforcement registry. This is
# the check on the checks, and it is what an absence-shaped failure
# needs, because absences never raise on their own.
def test_every_target_repo_enforces_the_check(manifest, enforcement):
targets = {repo for e in manifest["files"] for repo in e["targets"]}
for repo in sorted(targets):
assert repo in enforcement, (
f"{repo} receives vendored files but no suite checks them"
)
Find a rule in your team's documentation that is phrased as a location — a file that must exist at a path, a directory every service must have, a config key in a specific place. Ask who cannot comply with it structurally. Then rewrite it as the requirement it was trying to express, and add one enumeration that lists who is supposed to satisfy it and checks that they do.
Hint
Look at the newest participant first, and the most structurally unusual one second. The rule was almost certainly written when everyone had the same shape, and the exemption arrived silently with the first member that did not.
Progress
Progress is local-only — sign in to sync across devices.