Skip to content
C.W.K.
Lesson 02 of 04 · published

Measure Before Choosing

~15 min · latency, measurement, brain-slots, benchmarks

Level 0First Word
0 XP0/47 lessons0/13 achievements
0/100 XP to next level100 XP to go0% complete

A default the doc refused to guess

Dad's posture for apps shaped like Waystone is one brain slot per app. Spark's is pippa_actions: every turn that lives in a thread, from a photo's reading to an interpretation card, rides it. The architecture gave the live camera a conditional seat on that same slot. In its words, the live lane rides the practical slot unless a measurement argues for a separate one. Nobody picked the live lane's brain by taste. The document asked for a number first.

Ask exactly what the lane asks

scripts/measure-live-lane.py is that measurement. Its first rule is that it imports the lane's own prompt builders, live_prompt and live_system_prompt, from the engine. The question it times is the question the camera sends: four numbered Spanish signs, a moment whose target is Korean and whose situation is a street, persona Pippa, no token cap. It sends that straight to cwkPippa's sibling utility door for each brain:effort pair, three rounds each by default.

Two details keep it honest. It calls the door directly, so it writes nothing to any Spark corpus. A benchmark that left fake moments behind would pollute the very cache lesson 3 relies on. And a round counts only if every sign came back translated. The script parses the answer with the same parse_json_lines the engine uses, collects the numbers that carry a non-empty translation, and takes the median over complete rounds only. A fast incomplete answer does not count as fast.

The door answers only a loopback caller, so the script runs on the service host, next to cwkPippa.

What it found

Brain and effortMedianRounds (s)
Claude, low3.4 s3.6, 3.2, 3.4
GLM, low5.3 s5.4, 2.9, 5.3
Claude, high (the practical slot)5.9 s5.9, 4.4, 6.4
ChatGPT, low8.4 s8.9, 8.4, 8.2
Grok, low12.9 s23.8, 11.8, 12.9
Kimi, low16.0 s67.0, 16.0, 10.4

Read the rounds, not only the medians. Kimi's median is 16 s, and one of its three rounds took 67 s. On a camera, the slow round is the one Dad feels. Effort mattered as much as the brain: the same Claude was 5.9 s at the high effort the practical slot ran, and 3.4 s at low. Effort was the cheapest knob on the table.

For scale, a photo's practical turn on the practical slot showed its first line about 23 s after the request and finished twelve results at 32 s, most of it the brain thinking before it writes. A threaded turn also pays the full conversation load, which the architecture puts at a floor of about 5 s. The live lane skips that by design: one short call, no conversation.

Two slots, one owner

The measurement argued, so the lane got a second slot. brain_control.py declares both: pippa_actions in the main category and live_lane in the utility category, each with a description a person can read on the family Brains surface. INITIALS is only the first declaration. The kit adopts it once, at first registration. After that the family control plane is the only authority, and Dad moves either slot from the Brains surface like any other app's.

Notice where the evidence went. The live slot's description carries the measurement itself, Claude low against Claude high, so whoever opens the Brains surface sees why the default is what it is. The architecture document carries the full table, and the commit that created the slot carries the medians in its message. The reason travels with the setting.

The engine resolves the live slot on every request, with a cheap revision probe first, so his change applies to the next batch with no deploy. If the control plane cannot be reached, the kit serves the last assignment it saw, marked stale. With no such copy, the request gets no brain and says so (lesson 3). High-quality translation (Track 7) picks brains per job and never touches either slot. The interpreter's fast draft (Track 6) later borrowed the same live_lane slot, for the same reason: seconds show.

Principle: When a design says "unless a measurement argues otherwise", write the measurement as code that asks the real question, and keep it in the repository beside what it decided.

The brain-slot plane itself, with its catalog and pickers, belongs to the family, and Sibling Kit Quest shows how a shared piece like this reaches every app as vendored code. Pippa Quest covers what cwkPippa does behind the utility door.

Code

cwkSpark, scripts/measure-live-lane.py: one measured call, asking what the lane asks (verbatim, lines 35-71)·python
SAMPLE = ["Salida de emergencia", "Farmacia de guardia", "Prohibido fumar", "Cerrado por vacaciones"]
MOMENT = {"id": "measurement", "target_language": "ko", "scene_language": "es", "situation": "street"}
DEFAULT_BRAINS = "claude:high,claude:low,chatgpt:low,kimi:low,glm:low,grok:low"


async def one_call(brain: str, effort: str) -> dict:
    items = [{"key": str(index), "text": text} for index, text in enumerate(SAMPLE)]
    body = {
        "prompt": live_prompt(items),
        "system_prompt": live_system_prompt(MOMENT, "es"),
        "brain": brain,
        "effort": effort,
        "max_tokens": 0,
        "persona_soul_id": SOUL_ID,
        "source": "spark-live-measure",
    }
    started = time.monotonic()
    try:
        envelope = await kit_pippa.utility_complete(settings.pippa_base, body, timeout=180)
    except kit_pippa.PippaClientError as exc:
        envelope = {"ok": False, "result": "", "error": str(exc)}
    elapsed = time.monotonic() - started
    numbers = set()
    for obj in parse_json_lines(str(envelope.get("result") or "")):
        try:
            if str(obj.get("translation") or "").strip():
                numbers.add(int(obj.get("n")))
        except (TypeError, ValueError):
            continue
    return {
        "seconds": round(elapsed, 2),
        "translated": len(numbers & set(range(1, len(SAMPLE) + 1))),
        "ok": bool(envelope.get("ok")),
        "model": envelope.get("model") or "",
        "effort": envelope.get("effort") or effort,
        "error": envelope.get("error"),
    }
cwkSpark, backend/brain_control.py: two slots and their first declaration (verbatim, lines 52-86)·python
APP_ID = "spark"
SLOT_ID = "pippa_actions"
LIVE_SLOT_ID = "live_lane"

SLOTS: tuple[dict[str, str], ...] = (
    {
        "slot_id": SLOT_ID,
        "label": "Pippa actions",
        "category": "main",
        "description": (
            "The brain behind every Spark turn in a thread: photo and text "
            "translation, interpretation cards and relays, and explanations. "
            "High-quality versions pick their brains per job."
        ),
    },
    {
        "slot_id": LIVE_SLOT_ID,
        "label": "Live camera",
        "category": "utility",
        "description": (
            "The brain behind the live camera's one-shot translations through "
            "the utility door, where seconds show. Measured 2026-09-25 on four "
            "signs: Claude at low effort 3.4 s median, at high 5.9 s."
        ),
    },
)

# First registration adopts this; after that central is the only authority
# and Dad changes it from the family Brains surface. Spark has no legacy
# device picker to import, so the declaration names the family's canonical
# brain and lets its effort inherit (the Anvil shape).
INITIALS: dict[str, dict[str, str]] = {
    SLOT_ID: {"brain": "claude", "reasoning_mode": "inherit"},
    LIVE_SLOT_ID: {"brain": "claude", "effort": "low"},
}
cwkSpark, clients/apple/SparkMobile/Sources/EngineClient.swift: the phone's live policy, and the default transport every call without its own uses (verbatim, lines 36-41)·swift
    /// The live lane waits for Pippa's one short call (measured 3-6 s).
    static let livePolicy = HTTPSessionPolicy(requestTimeout: 60, resourceTimeout: 75, byteBudget: 1 << 20)
    static let streamPolicy = HTTPSessionPolicy(requestTimeout: 120, resourceTimeout: 900, byteBudget: 8 << 20)

    init(endpoint: EngineEndpoint,
         transport: @escaping HTTPTransport = HTTPTransports.urlSession(policy: EngineClient.jsonPolicy),

External links

Exercise

Pick one slow dependency in software you are building (a model call, a search API, a remote database query) and write a measurement script for it. Import the real request builder instead of retyping the request. Vary one knob (provider, model, effort, region), run at least three rounds each, count a round only when the answer is complete by the same parser production uses, and print every round beside the median. Make sure the script writes nothing to production state. Then choose a default from the table and commit the table next to the choice.
Hint
If the script builds its own copy of the request, you are timing a question nobody asks. Keep the incomplete rounds visible as a separate count instead of dropping them silently, and look for one outlier round per knob: the median hides it, and the person waiting at the other end feels it. Finally, check that the timeout you chose from the table is actually passed at the call site.

Progress

Progress is local-only — sign in to sync across devices.
← PreviousA Scanner That Keeps Track of TextNext →Text Already Met Is Instant
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign in — Please sign in to comment.

No comments yet — be the first.