Small models — anything under about 4B — struggle with long, layered system prompts. They follow them poorly, drift, and often output meta-commentary instead of doing the task. Large models (32B+) handle dense system prompts naturally. Your adapter has to know which size it's talking to and adapt the system prompt accordingly.
Mini mode
"Mini mode" is the convention: when the active model is small, swap to a stripped-down system prompt. Keep the plan, drop the philosophy. Keep the format, drop the tone calibration.
Where the metadata comes from
/api/tags returns each installed model's details.parameter_size ("7B", "32B", "1.5B"). Parse that, classify into mini / standard / large, and route prompts accordingly.
Don't reinvent classification
Three buckets is enough: mini (under 4B), standard (4B to under 15B), large (15B and up). More buckets = more if/else chains for marginal benefit.
Code
Mini-mode-aware selector·python
class OllamaModelSelector:
"""Smart model selection layered over OllamaAdapter."""
def __init__(self, adapter: OllamaAdapter):
self.adapter = adapter
async def installed(self) -> list[dict]:
r = await self.adapter.client.get(f"{self.adapter.base_url}/api/tags")
r.raise_for_status()
return r.json().get("models", [])
def classify(self, model_info: dict) -> str:
size = model_info.get("details", {}).get("parameter_size", "")
try:
params_b = float(size.replace("B", "").strip())
except ValueError:
params_b = 0.0
# Ollama reports the marketing size, not a round number: llama3.2:latest
# says "3.2B" and qwen2.5:7b says "7.6B". Cut above the class, not at it,
# or the canonical 3B model lands in the wrong bucket.
if params_b < 4:
return "mini"
if params_b < 15:
return "standard"
return "large"
def system_prompt_for(self, klass: str) -> str:
if klass == "mini":
# Small models: stripped down — plan + format only
return (
"You are a helpful assistant. Be concise. "
"If asked to use a tool, call it directly without commentary."
)
# standard + large can handle the full prompt
return (
"You are a knowledgeable AI assistant with expertise in software "
"engineering, data analysis, and creative work. Think step by step. "
"Provide detailed, accurate responses. When using tools, explain "
"your plan in one sentence before calling them."
)
async def system_prompt_for_active(self) -> str:
"""Look up the active model's class and return the right prompt."""
models = await self.installed()
info = next((m for m in models if m["name"] == self.adapter.model), {})
return self.system_prompt_for(self.classify(info))
Implement OllamaModelSelector. Run classify() on every installed model on your machine and print the buckets. Send the same prompt with mini-mode and standard-mode system prompts to a 1B and a 7B model. Note when the 1B model behaves better with mini mode.
Progress
Progress is local-only — sign in to sync across devices.