C.W.K.
Stream
Lesson 03 of 05 · published

Model Selection & Mini Mode

~18 min · adapters, selection

Level 0Downloader
0 XP0/41 lessons0/11 achievements
0/120 XP to next level120 XP to go0% complete

The size-vs-prompt mismatch

Small models (1B–3B) struggle with long, layered system prompts. They follow them poorly, drift, and often output meta-commentary instead of doing the task. Large models (32B+) handle dense system prompts naturally. Your adapter has to know which size it's talking to and adapt the system prompt accordingly.

Mini mode

"Mini mode" is the convention: when the active model is small, swap to a stripped-down system prompt. Keep the plan, drop the philosophy. Keep the format, drop the tone calibration.

Where the metadata comes from

/api/tags returns each installed model's details.parameter_size ("7B", "32B", "1.5B"). Parse that, classify into mini / standard / large, and route prompts accordingly.

Don't reinvent classification

Three buckets is enough: mini (≤3B), standard (4–14B), large (≥15B). More buckets = more if/else chains for marginal benefit.

Code

Mini-mode-aware selector·python
class OllamaModelSelector:
    """Smart model selection layered over OllamaAdapter."""

    def __init__(self, adapter: OllamaAdapter):
        self.adapter = adapter

    async def installed(self) -> list[dict]:
        r = await self.adapter.client.get(f"{self.adapter.base_url}/api/tags")
        r.raise_for_status()
        return r.json().get("models", [])

    def classify(self, model_info: dict) -> str:
        size = model_info.get("details", {}).get("parameter_size", "")
        try:
            params_b = float(size.replace("B", "").strip())
        except ValueError:
            params_b = 0.0
        if params_b <= 3:
            return "mini"
        if params_b <= 14:
            return "standard"
        return "large"

    def system_prompt_for(self, klass: str) -> str:
        if klass == "mini":
            # Small models: stripped down — plan + format only
            return (
                "You are a helpful assistant. Be concise. "
                "If asked to use a tool, call it directly without commentary."
            )
        # standard + large can handle the full prompt
        return (
            "You are a knowledgeable AI assistant with expertise in software "
            "engineering, data analysis, and creative work. Think step by step. "
            "Provide detailed, accurate responses. When using tools, explain "
            "your plan in one sentence before calling them."
        )

    async def system_prompt_for_active(self) -> str:
        """Look up the active model's class and return the right prompt."""
        models = await self.installed()
        info = next((m for m in models if m["name"] == self.adapter.model), {})
        return self.system_prompt_for(self.classify(info))

External links

Exercise

Implement OllamaModelSelector. Run classify() on every installed model on your machine and print the buckets. Send the same prompt with mini-mode and standard-mode system prompts to a 1B and a 7B model. Note when the 1B model behaves better with mini mode.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.