Verify the Evidence

A green exit is not the same as a verified claim. Start by binding the observation to the archived submission manifest and, for repository tasks, the exact frozen-base manifest. Then read the command, network posture, exit status, clipped-output flag, and actual output.

Integrity proves which bytes were exercised. Relevance asks whether that command can observe the behavior being claimed. A passing unit suite does not prove a browser layout; a screenshot at one pinned viewport does not prove accessibility or every screen size.

Unavailable evidence is an honest gap, not a mandatory gate failure. The evidence pack records that the sandbox or screenshot facility could not run, or a command started but timed out. Judges weigh that absence alongside the artifact and other evidence; they do not replace it with a contestant's unsupported success story.

Negative and partial outcomes deserve exact language: “command exited 1 after these checks,” “output was clipped,” or “screenshot unavailable.” State the strongest supported claim, the exclusions, and the remaining uncertainty.

Verify identity, execution, relevance, and scope. A trustworthy pack makes failure and absence as legible as success.

Code

Auditing evidence — green is not a claim·python

def audit(row, manifest_shas, claim_needs):
    """Three questions in order. Miss one and the claim does not stand."""
    findings = []
    if row.get("sha") not in manifest_shas:
        findings.append("integrity: these are not the declared bytes")
    if "unavailable" in row:
        findings.append(f"absence: {row['unavailable']} — an honest gap")
    elif row.get("exit") != 0:
        findings.append(f"execution: exited {row['exit']} — itself an observation")
    if claim_needs not in row.get("observes", []):
        findings.append(f"relevance: this command cannot see {claim_needs}")
    return findings or ["this claim is supported"]


shas = {"aa11"}

passing = {"sha": "aa11", "exit": 0, "observes": ["unit"]}
print(audit(passing, shas, claim_needs="unit"))

# Passed, but watched the wrong thing — the most common misreading.
print(audit(passing, shas, claim_needs="browser_layout"))

missing = {"sha": "aa11", "unavailable": "no sandbox", "observes": ["unit"]}
print(audit(missing, shas, claim_needs="unit"))

Verify the Evidence

Verify the Evidence

Code

External links

Exercise

Progress

Comments 0