Two Vantages Running One Method Are One Instrument

The Rule That Was Right, Applied Wrong

A corpus-wide sweep checked every external reference for liveness and produced a worklist of dead ones. Earlier experience had already taught a real lesson here — a single probe, from one host, at one moment, is evidence about the probe rather than about the target. So the sweep ran from two machines on different networks and listed only references that failed from both.

That rule is correct, and six of the one hundred and ninety entries on the resulting worklist were live.

What the Two Vantages Actually Shared

The check sent a metadata-only request first and only retried with a full request on a couple of specific refusal codes. A handful of targets answer that metadata request with a not-found and the full request with success — a real and not-uncommon server behavior. Those targets read as dead from every vantage, forever.

Two vantages de-noise network variance. This was method variance, and the two vantages shared the method completely. The check built specifically to catch false positives could not, structurally, catch this class of them.

The general form is worth carrying: ask what your independent confirmations have in common before counting them as two. Two people using the same checklist are one checklist. Two services querying the same upstream are one upstream. Agreement is only evidence when the things agreeing could have disagreed.

What Independence Actually Looks Like

The fix in this case was not more vantages; it was a differential. Send both request forms to the surviving dead set and compare. That is cheap, it targets the exact failure, and it belongs at diagnosis time rather than after a repair has been proposed.

And when you need to answer a harder question — did this reference ever exist, or has it merely moved — the instrument has to come from a different direction entirely. Query the source repository's own history for that path: a path that never appears in any commit was never there, which distinguishes a reorganization from a fabrication. One caution that generalizes: run a control alongside it. An absence in an index is evidence about the index until you have shown the index has coverage for things you know exist.

Agreement between instruments that share a method is not independent evidence. Before treating two confirmations as two, write down what they have in common — the method, the library, the upstream, the assumption. Whatever is on that list is the thing neither of them can see.

Code

The differential that the two-vantage rule could not replace·python

def liveness(url):
    """A metadata-only request and a full request can DISAGREE, and
    the disagreement is a property of the server, not the network.
    Running this from two machines does not help: both machines
    send the same two requests and get the same two answers.
    """
    meta = probe(url, method="metadata-only")
    full = probe(url, method="full")
    if meta.status != full.status:
        return ("alive" if full.ok else "unverifiable",
                f"metadata={meta.status} full={full.status} - the"
                " check disagrees with itself")
    if full.status in (404, 410):
        return "dead", f"{full.status} on both forms"
    if full.ok:
        return "alive", ""
    return "unverifiable", f"{full.status} - not proof of absence"


# THREE VERDICTS, NOT TWO. "unverifiable" is what stops a blocked
# request or a bot wall from being filed as rot. A disagreement
# between the two forms can never be filed as DEAD - it resolves
# to alive when the full request succeeds and to unverifiable
# otherwise, and never to the answer you prefer.


# For the harder question - did this ever exist, or did it move?
#   ask the source's OWN history for the path:
#     GET /repos/{owner}/{repo}/commits?path=<path>
#   zero commits  -> the path was never on the default branch
#   but ALWAYS run a control: a sibling path you KNOW exists.
#   an empty answer is evidence about the index until the control
#   proves the index has coverage.

Exercise

Take a check your team runs from more than one place and list everything the runs have in common: the library, the request shape, the timeout, the parsing, the assumption about what a failure means. Then design one alternative that shares as little of that list as possible and run both on the same input. The disagreements are the defects your redundancy was never able to see.

Hint

The shared item that hides best is the parsing. Two probes from two continents through two networks will still both call a redirect-to-a-login-page a success if they both only look at the status code, and that failure is invisible until something reads the body.

The Rule That Was Right, Applied Wrong

What the Two Vantages Actually Shared

What Independence Actually Looks Like

Code

External links

Exercise

Progress

Comments 0