Stateless Sessions and Remote Ollama

Ollama is stateless

Ollama doesn't remember between requests. There's no session id, no server-side history. To have a conversation, you replay the whole message history every turn. Compare to Claude Agent SDK (which holds a persistent subprocess per session) — that's the exception, not the rule. Most LLM APIs are stateless; Ollama is the common case.

What you build

A thin OllamaSession on top of the adapter that maintains the message list, appends each user/assistant turn, and replays on every send. The adapter doesn't know it exists — sessions are an upper layer.

Remote Ollama

Set OLLAMA_HOST=0.0.0.0:11434 on the server Mac to expose the daemon on the network. Then your laptop can point the adapter at http://server-mac.local:11434 (or a Tailscale IP) and offload inference to bigger hardware. Same API, different base URL.

The Pippa fleet pattern

cwkPippa's office Mac runs Ollama with the bigger 70B-class models loaded. Other Macs in the fleet point their Ollama adapters at the office's Tailscale IP for heavy inference and run small models locally for fast small tasks. The choice is per-call, set in the adapter's base_url.

Code

Session wrapper around the adapter·python

class OllamaSession:
    """Maintain conversation state for stateless Ollama."""

    def __init__(self, adapter: OllamaAdapter, system_prompt: str | None = None):
        self.adapter = adapter
        self.history: list[dict] = []
        if system_prompt:
            self.history.append({"role": "system", "content": system_prompt})

    async def send(self, user_message: str) -> str:
        self.history.append({"role": "user", "content": user_message})
        full = ""
        async for chunk in self.adapter.stream(self.history):
            if chunk.done:
                break
            full += chunk.content
        self.history.append({"role": "assistant", "content": full})
        return full

    def clear(self) -> None:
        """Reset, keeping only the system prompt."""
        self.history = [m for m in self.history if m["role"] == "system"]

Remote Ollama via Tailscale·python

# On the server Mac (one-time):
#   launchctl setenv OLLAMA_HOST 0.0.0.0:11434
#   (then restart Ollama)
# This exposes the daemon on all interfaces. Use Tailscale or another VPN —
# never a public IP without a real auth layer in front.

LOCAL  = OllamaAdapter(base_url="http://localhost:11434",
                        model="qwen2.5:3b")          # fast small tasks
REMOTE = OllamaAdapter(base_url="http://100.x.x.x:11434",
                        model="qwen2.5:72b")         # heavy reasoning

async def smart_call(prompt: str, heavy: bool):
    a = REMOTE if heavy else LOCAL
    async for chunk in a.stream([{"role": "user", "content": prompt}]):
        if chunk.done:
            break
        print(chunk.content, end="", flush=True)

Stateless Sessions and Remote Ollama

Ollama is stateless

What you build

Remote Ollama

The Pippa fleet pattern

Code

External links

Exercise

Progress

Comments 0