C.W.K.
Stream
Lesson 02 of 05 · published

OllamaAdapter Implementation

~26 min · adapters, ollama

Level 0Downloader
0 XP0/41 lessons0/11 achievements
0/120 XP to next level120 XP to go0% complete

What this adapter does

Translates Ollama's NDJSON streaming to the universal StreamChunk. Handles message content, tool calls, the final done frame with timing. Implements health check (any HTTP 200 from /api/version) and model listing (/api/tags).

Translation map

Ollama fieldStreamChunk field
message.contentcontent
message.tool_callstool_calls (passed through; already parsed dicts)
done: truedone = True + tokens_per_sec
(absent)thinking (only used by reasoning models that emit it)

Where to handle errors

Inside the adapter's stream(). Connection refused → OllamaUnavailable (caught by orchestrator and triggers fallback). Read timeout → ModelLoadingTimeout. Malformed JSON line → OllamaProtocolError. Each one carries enough context to log meaningfully.

Code

Full OllamaAdapter·python
import httpx, json
from typing import AsyncIterator, Any

class OllamaUnavailable(Exception): ...
class OllamaProtocolError(Exception): ...

class OllamaAdapter(AIAdapter):
    def __init__(self, base_url: str = "http://localhost:11434",
                 model: str = "qwen2.5:7b"):
        self.base_url = base_url.rstrip("/")
        self.model = model
        self.client = httpx.AsyncClient(timeout=None)

    async def stream(
        self,
        messages: list[dict],
        tools: list[dict] | None = None,
        **opts: Any,
    ) -> AsyncIterator[StreamChunk]:
        payload: dict = {"model": self.model, "messages": messages, "stream": True}
        if tools:
            payload["tools"] = tools
        if opts:
            payload["options"] = opts

        try:
            async with self.client.stream("POST", f"{self.base_url}/api/chat",
                                           json=payload) as r:
                r.raise_for_status()
                async for line in r.aiter_lines():
                    if not line:
                        continue
                    try:
                        chunk = json.loads(line)
                    except json.JSONDecodeError as e:
                        raise OllamaProtocolError(f"bad NDJSON: {e}") from e

                    msg = chunk.get("message", {})

                    if chunk.get("done"):
                        ec = chunk.get("eval_count", 0)
                        ed = chunk.get("eval_duration", 1) or 1
                        tps = ec / (ed / 1e9) if ed else 0.0
                        yield StreamChunk(done=True, tokens_per_sec=tps,
                                          model=self.model, raw=chunk)
                        return

                    if msg.get("tool_calls"):
                        yield StreamChunk(tool_calls=msg["tool_calls"], raw=chunk)
                        continue

                    yield StreamChunk(content=msg.get("content", ""), raw=chunk)
        except httpx.ConnectError as e:
            raise OllamaUnavailable("Ollama daemon not reachable") from e

    async def health_check(self) -> bool:
        try:
            r = await self.client.get(f"{self.base_url}/api/version", timeout=2.0)
            return r.status_code == 200
        except Exception:
            return False

    async def list_models(self) -> list[str]:
        r = await self.client.get(f"{self.base_url}/api/tags")
        r.raise_for_status()
        return [m["name"] for m in r.json().get("models", [])]

External links

Exercise

Implement OllamaAdapter in your own project with the three methods. Test stream() end-to-end with a simple chat. Test tool_calls translation by sending a tool definition. Test health_check() with the daemon stopped — should return False fast (under 3 seconds).

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.