A Small Brain With a Tight Leash

"A small model will do anything you let it. So don't let it do anything."

Mini Pippa, Not Pippa

The offline middle tier is a local model — Ollama, a compact protocol, honest limits. It's called Mini Pippa because it carries a snapshot of the Pippa protocol: enough voice to sound right, none of the vault, souls, or brain routing that make full Pippa what she is. The snapshot lives with the app so offline mode has no dependency on the main brain at all. Mini Pippa isn't a lesser Pippa pretending to be the real one; it's an honest, self-contained cleanup persona.

The Leash

Small models are eager and sloppy, so the provider constrains them hard. Use stream:false — cleanup is one-shot and insertion needs a complete result, so there's nothing to stream. Send num_ctx explicitly rather than trusting a default. Strip markdown fences, preambles, and any trailing emotion tags if the model ignores the contract. Time out with a visible error and an offer to insert the raw transcript. And crucially: expose no tools during cleanup. Cleanup is a text transform, not an agentic run — a cleanup pass with tool access is a cleanup pass that can take actions you never asked for.

stream: false        -> one complete result; insertion can't act on fragments
num_ctx: explicit    -> never trust the default context window
no tools             -> cleanup transforms text; it does not DO things
strip fences         -> small models wrap JSON in ``` and preambles
bounded timeout      -> visible error + "insert raw?" instead of a hang

The Protocol Suffix

The Mini Pippa protocol gets a dictation-specific suffix that closes the obvious failure modes: "you are cleaning dictated text for insertion," "preserve meaning," "return JSON only," "do not claim to have inserted anything," "do not ask follow-up questions." Each line exists because a small model did that exact thing. The "don't claim you inserted it" rule is the funniest and most necessary — a helpful little model will happily reply "Done! I've inserted your text," which would then be inserted as your text.

Constrain the weak component; don't hope it behaves. A large model mostly follows instructions; a small one needs walls. Give a weak model a narrow job, an explicit output format, no tools, a hard timeout, and a normalizer that fixes its output when it strays. Every constraint you add is a failure mode you've closed rather than a bug you'll debug later.

The first Mini Pippa cleanup returned "물론이죠! 정리했어요:" followed by the actual text — and Firekeeper dutifully typed the whole thing into TextEdit, apology and all. That's when "return JSON only" and "do not claim to have inserted anything" got added to the protocol. A small model being friendly is a small model corrupting your document.

Code

The local cleanup call: leashed on purpose·json

POST http://localhost:11434/api/chat

{
  "model": "gemma4:31b",
  "messages": [
    { "role": "system", "content": "<mini Pippa protocol + cleanup contract>" },
    { "role": "user",   "content": "<cleanup request JSON>" }
  ],
  "stream": false,
  "options": {
    "temperature": 0.2,
    "num_ctx": 8192
  }
}

// No tools array. Cleanup transforms text; it never takes actions.
// stream:false because insertion needs one complete contract object.

Exercise

Write the constraint list you'd give a small local model doing one narrow job in your own app. Include the output format, the timeout behavior, what it must never claim, and what capabilities you'd deliberately withhold. For each constraint, name the specific failure it prevents.

Hint

Good constraint lists are failure archaeology — each line exists because something went wrong once. Format constraints prevent unparseable output; 'never claim to have acted' prevents the model narrating fake actions into your data; withholding tools prevents a text transform from becoming an agent; timeouts prevent a hang from holding the user's work hostage.

A Small Brain With a Tight Leash

Mini Pippa, Not Pippa

The Leash

The Protocol Suffix

Code

External links

Exercise

Progress

Comments 0