Skip to content
C.W.K.
Stream
Lesson 02 of 04 · published

Runner, Not Story

~11 min · evidence-runner, commands, observation, reproducibility

Level 0Cold Iron
0 XP0/36 lessons0/12 achievements
0/100 XP to next level100 XP to go0% complete

Runner, Not Story

A contestant's “tests pass” remains part of its report. Anvil's evidence runner makes a separate deterministic observation. The composer declares up to four command strings, and after archive Anvil runs each with the working directory at a scratch copy of that submission's stored tree.

On macOS the runner uses sandbox-exec: writes stay in scratch; reads under Dad's home and mounted volumes are denied except for the engine's fixed conda toolchain path; credentials are scrubbed from the environment; each command has a wall-clock cap; and network follows the run's open or hermetic posture. The archived judged bytes are never mutated.

A completed command row carries cmd, exit, seconds, output, and truncated. If the sandbox or executable is unavailable, the row carries cmd and unavailable with a reason. A timeout is also filed as unavailable, but may include partial output and a truncated flag because execution did begin.

Every judge receives the same rows in the evidence pack. Non-zero exit, timeout, missing sandbox, and success remain distinguishable observations. Anvil does not rewrite the submission, remove it silently, or turn infrastructure absence into a fabricated success.

Run declared commands against an archived copy. The runner records its exact observation; the judge decides what it means.

Code

Real runner rows — three distinct observations·json
[
  { "cmd": "pytest -q", "exit": 1, "seconds": 4.2,
    "output": "1 failed, 9 passed", "truncated": false },

  { "cmd": "npm test",
    "unavailable": "sandbox-exec unavailable" },

  { "cmd": "slow-check",
    "unavailable": "timed out after 60s",
    "output": "partial output…", "truncated": false }
]

External links

Exercise

Write the exact pack row for one non-zero command, one missing sandbox, and one timed-out command with partial output.
Hint
Use cmd/exit/seconds/output/truncated and unavailable as a reason-string key.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.