When the Reviewer Is Unavailable

A Rate Limit Is a Scheduling Fact, Not a Verdict

You ask for a specific reviewer and get nothing back — a quota refusal, a crash, a spinner that never resolves. The dangerous move is to record that as a review round, because now the log says the artifact was read and it was not. The other dangerous move is to stop the pipeline, because unavailability is a property of the moment rather than of the work.

For the five weeks it lived, the answer was a ring: an ordered rotation of reviewers, entered at whichever one was requested, walked around exactly once. It has since been retired — the note at the end of this lesson carries the ruling — and what follows here is the measurement half, which survives intact. Every hop is logged with the reason the previous one was skipped. If the ring closes without a reviewer, that is the point where a human decides — and the decision is usually to use a reviewer somebody had informally excluded, which is why the ring has to be written down rather than assembled in the moment. Written down does not mean everyone: a member that misbehaves when named — one that runs alone instead of joining the rotation — is excluded on purpose, and the exclusion is part of the record.

Measuring Availability Without Fooling Yourself

This is where it gets subtle, because every obvious signal lies in a different direction.

Byte count lies. One failure produced over seven thousand characters on the wire and displayed as a handful of spinner frames — control codes and carriage returns, not an answer. Strip the escape sequences and measure the rendered text.

Exit code lies too. Another produced eight thousand characters, exited non-zero, and carried a quota message in its tail. Long, and dead. So a crash whose output matches a limit pattern counts as unavailable at any length.

A short clean exit is not an answer. Exiting zero having said almost nothing means the tool ran and produced no review.

A long clean answer is never second-guessed, whatever words it contains. This one is load-bearing in the other direction: a genuine, substantial review of rate-limit handling code would trip any naive keyword scan, and killing it would be the worst possible false positive.

The Shape That Stays Undetected on Purpose

There is a failure this cannot catch: a reviewer that returns just over the displayed-length floor in plain narration, exits zero, and never uses limit vocabulary. It is not a crash, not a quota message, and not short enough for the length test to convict. Every mechanical detector anyone has proposed for it re-creates the false positive above.

So it is left to the session's own judgment, deliberately and with that written down. The reason to name an undetectable case in a contract is that the alternative is a team believing the detector covers it.

Tighten a detector from recorded failure output, never from imagination. Every hop should log the tail of what it actually saw, so the next person extends the patterns from real specimens. Patterns invented in advance match nothing and fail silently, which is worse than having none — because you believe you are covered.

Code

Availability, measured on what a human would have seen·python

import re

ANSI = re.compile(r"\x1b\[[0-9;?]*[a-zA-Z]")
LIMIT = re.compile(r"usage limit|rate limit|quota|too many requests|"
                   r"429|403", re.I)
MIN_DISPLAYED = 400          # chars of RENDERED text


def displayed(raw: str) -> str:
    """What a person would have seen. A spinner sends thousands of
    characters and shows a few lines; measure what was shown."""
    text = ANSI.sub("", raw)
    # a carriage return rewrites the line - keep only the last write
    return "\n".join(seg.split("\r")[-1] for seg in text.split("\n"))


def unavailable(raw: str, exit_code: int) -> str | None:
    shown = displayed(raw)
    if exit_code != 0 and LIMIT.search(shown[-2000:]):
        return "limit-shaped crash"      # dead at ANY length
    if exit_code != 0 and looks_like_a_limit(shown):
        return "crash"      # non-zero ALONE is not enough:
                             # a crash with a real answer above it
                             # is still an answer
    if len(shown.strip()) < MIN_DISPLAYED:
        return "clean exit, no answer"   # ran, reviewed nothing
    return None                          # a long clean answer is
                                         # NEVER second-guessed -
                                         # it may be a real review
                                         # OF rate-limit code


# UNDETECTABLE ON PURPOSE: a few hundred chars of plain narration,
# exit 0, no limit vocabulary. Not a crash, not a quota message,
# not short enough to convict. Every mechanical rule proposed for
# it re-creates the false positive above, so it is left to the
# session's judgment - and that is written into the contract so
# nobody believes the detector covers it.

Exercise

Find a place in your system where an external dependency's failure is currently recorded as a result. Write down what your logs would look like in both cases — genuine empty result versus dependency down — and check whether anything distinguishes them. Then add the distinguishing field, and make the failure path log the raw tail of what it received rather than a summary.

Hint

The raw tail is the part people leave out because it is ugly, and it is the only part that lets a future maintainer extend the detection. A log line saying "dependency unavailable" teaches nothing; the same line with two hundred characters of what actually came back is what the next fix is built from.

A Rate Limit Is a Scheduling Fact, Not a Verdict

Measuring Availability Without Fooling Yourself

The Shape That Stays Undetected on Purpose

Code

External links

Exercise

Progress

Comments 0