The Owner File Fails Closed

A Small File That Moves Every Call

The fence from the last lesson stops raw writers cold. That raises an awkward question: Coop's ledger had many honest callers before the engine existed: every Pippa session, every helper script, Almanac's bridge, all of them speaking python -m tools.coop and importing its functions. Rewriting each caller on cutover day would have been a second migration stacked on the first. So the cutover kept the database path and the call surface exactly as they were, and changed one thing: a small JSON file beside the store, <store>.owner.json.

That file says mode: service. It names the database it governs, the owner's URL and a private credential, and records when the transfer happened and where the pre-transfer backup sits. The compatibility client, the pinned copy of Coop's domain code that cwkPippa carries as tools.coop, looks for it on every call. That is why running Pippa turns did not have to be killed at cutover: the next call simply found the file.

Routing, and the One Place Code Runs Locally

Every domain operation (add_todo, take_todo, list_history and the rest) is wrapped by routed. With an owner file present, the wrapper never runs the function. It ships the method name, arguments and caller metadata to the owner's compatibility endpoint over HTTP. The only place the same domain code runs locally is inside the owner's own request context: when the engine executes an operation, the context is active, the wrapper steps aside, and the function runs on the engine's borrowed, fenced connection. One codebase, two roles, decided per call.

The lowest door is shut too. connect(), the helper anything would use to open the store directly, checks for an owner first and refuses outright: Direct SQL is disabled; use the Coop service API. So on the machine where the store lives, even a read goes through the service. A bad owner file does not get guessed around either. A mode other than service, or a file naming a different database, raises Invalid Coop owner configuration. A file that is not valid JSON, or that says service but leaves out the database, never reaches that message: it escapes as Python's own JSONDecodeError or KeyError. The words differ, the outcome does not. The exception leaves owner_config() before any call is routed or any connection opened, so nothing proceeds.

The Receipt Comes Before the Request

Before the client sends a single byte, it checks that the credential file is private (any group or world bit raises Coop credential must be private (0600)), mints a fresh request key, and writes a receipt to a private directory with state uncertain. The receipt holds the key, the method and a SHA-256 of the input, never bodies or credentials. It is created exclusively, set to 0600 and fsync'd. Only then does the HTTP call go out.

Why first? Because once a request crosses a process boundary, its fate is unknowable from this side until an answer comes back. A receipt written afterwards is lost exactly when you need it most, in a crash or a dropped connection between send and reply. Written before, it survives all of them. And for a write, the key it carries is the very key the engine files the command under (the next lesson shows what the engine does with it).

Failing Closed, in Exact Words

What went wrong	The client says	Receipt
Owner unreachable (network or socket error)	`Coop owner unavailable; no local write attempted; reconcile request <key>`	stays uncertain
Owner errored, no domain verdict (5xx)	`Coop owner failed (HTTP n); reconcile request <key>`	stays uncertain
Refused before running (4xx, no verdict)	`Coop refused the request (HTTP n): ...`	rejected
The domain said no	the domain's own error, re-raised	rejected
Success	the result	committed

Read the first row slowly, because every clause is a promise. No local write attempted: the client never falls back to opening SQLite itself, since a fallback writer would be precisely the second writer the fence exists to stop. Reconcile request: the client does not retry. The code says so in a comment, do not retry a request with an uncertain HTTP outcome, because the request may already have committed. It hands you the key instead. Coop's own test kills the service mid-session and asserts that the next add_todo raises with no local write in the message.

The receipt states had their own bug. Until 2512cf9 (2026-09-29), a call the service refused could still be recorded as committed; the hardening pass made rejected a state of its own. Settled receipts older than a day are pruned at most once an hour (the directory had grown to 537 files in three days), and uncertain ones are kept until someone reconciles them.

A fallback is a second writer in disguise. When the owner is gone, the honest answer is a loud error that names what to reconcile. Any quiet local write would buy availability by giving up the only guarantee the store had.

Code

Coop's compatibility client: owner discovery, routing, the direct-SQL refusal (trimmed excerpt)·python

# Coop's compatibility client: owner discovery, routing, the direct-SQL refusal
# (trimmed excerpt)

def owner_config():
    path = Path(os.environ.get("PIPPA_COOP_OWNER_FILE", str(db_path()) + ".owner.json"))
    if not path.exists():
        return None
    config = json.loads(path.read_text())
    if config.get("mode") != "service" or Path(config["database"]).resolve() != db_path().resolve():
        raise OwnerUnavailable("Invalid Coop owner configuration")
    return config


def routed(fn):
    @wraps(fn)
    def call(*args, **kwargs):
        config = None if context.active() else owner_config()
        return remote(fn.__name__, args, kwargs, config) if config else fn(*args, **kwargs)
    return call


def connect() -> sqlite3.Connection:
    borrowed = context.connection()
    if borrowed is not None:
        return borrowed
    if owner_config() is not None:
        raise OwnerUnavailable("Direct SQL is disabled; use the Coop service API")
    ...

Coop's compatibility client: one call to the owner (trimmed excerpt)·python

# Coop's compatibility client: one call to the owner (trimmed excerpt)

def remote(name, args, kwargs, config):
    token_path = Path(config["token_file"])
    if token_path.stat().st_mode & 0o077:
        raise OwnerUnavailable("Coop credential must be private (0600)")
    payload = {"method": name, "args": args, "kwargs": kwargs,
               "key": str(uuid.uuid4()),
               "metadata": {k: context.metadata(k) for k in ("id", "model", "reasoning", "session", "host")}}
    # Record the request ID before crossing the process boundary. Receipts contain
    # input hashes, not private bodies or credentials. Reconcile this ID on outage.
    receipt = {"key": payload["key"], "method": name, "state": "uncertain",
               "input_sha256": hashlib.sha256(json.dumps(payload, sort_keys=True, default=encode).encode()).hexdigest()}
    ...  # created exclusively, mode 0600, flushed and fsync'd before the request

    try:
        # Do not retry a request with an uncertain HTTP outcome.
        with urllib.request.urlopen(req, timeout=30) as response:
            result = json.loads(response.read(), object_hook=decode)
    except urllib.error.HTTPError as exc:
        ...  # no domain verdict: 4xx settles "rejected", 5xx stays uncertain
             # "Coop refused the request (HTTP {code}): ..."
             # "Coop owner failed (HTTP {code}); reconcile request <key>"
    except (OSError, ValueError) as exc:
        raise OwnerUnavailable("Coop owner unavailable; no local write attempted; reconcile request "
                               + payload["key"]) from exc
    ...
    settle("rejected" if "error" in result else "committed")

Exercise

Find a client in your own code that talks to a service and has a fallback path for when the service is down: a local cache it writes to, a queue file, a direct database connection. For that fallback, write down who reconciles its writes with the service's state afterwards, and what happens if the service actually received the original request. If nobody reconciles, sketch the fail-closed version: a receipt written before the call, a loud error naming the receipt, and no local write.

Hint

The dangerous fallbacks are the ones that make the write look successful. A cache that only serves reads during an outage is harmless; a cache that accepts writes has quietly made itself a second writer, and the reconciliation you never wrote is the bug.

A Small File That Moves Every Call

Routing, and the One Place Code Runs Locally

The Receipt Comes Before the Request

Failing Closed, in Exact Words

Code

External links

Exercise

Progress

Comments 0