NDJSON → SSE Proxy

Why you'd build this proxy

Most cloud-shaped chat UIs and SDKs (OpenAI Python SDK, Vercel AI SDK, LangChain streaming hooks) consume SSE. If you want to drop Ollama into one of those without rewriting the frontend, build a tiny proxy that converts NDJSON → SSE on the way out.

The shape of the conversion

Ollama emits {json}\n; SSE wants data: {json}\n\n.
OpenAI-compatible SSE clients expect a data: [DONE]\n\n sentinel at the end.
The chunk shape inside data: usually has to be reformatted into OpenAI's delta shape: {"choices": [{"delta": {"content": "..."}}]}.

Don't reinvent OpenAI compatibility

Ollama already exposes /v1/chat/completions as an OpenAI-compatible endpoint. If all you need is "drop-in OpenAI SDK", point the SDK at http://localhost:11434/v1 and skip the proxy. Build your own proxy only when you need transformations Ollama's compat endpoint doesn't do (custom auth, observability, request shaping).

Code

FastAPI proxy: NDJSON → SSE in OpenAI delta shape·python

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import httpx, json

app = FastAPI()

async def ollama_to_openai_sse(model: str, messages: list[dict]):
    """Bridge Ollama NDJSON to OpenAI-shaped SSE."""
    async with httpx.AsyncClient(timeout=None) as client:
        async with client.stream(
            "POST",
            "http://localhost:11434/api/chat",
            json={"model": model, "messages": messages, "stream": True},
        ) as r:
            async for line in r.aiter_lines():
                if not line:
                    continue
                chunk = json.loads(line)
                content = chunk.get("message", {}).get("content", "")
                done = chunk.get("done", False)

                payload = {
                    "id": "ollama-stream",
                    "object": "chat.completion.chunk",
                    "model": model,
                    "choices": [{
                        "index": 0,
                        "delta": {"content": content} if content else {},
                        "finish_reason": "stop" if done else None,
                    }],
                }
                yield f"data: {json.dumps(payload)}\n\n"
                if done:
                    yield "data: [DONE]\n\n"
                    return

@app.post("/v1/chat/completions")
async def chat_completions(req: dict):
    return StreamingResponse(
        ollama_to_openai_sse(req["model"], req["messages"]),
        media_type="text/event-stream",
    )

Or just use Ollama's built-in OpenAI compat·bash

# No proxy needed if drop-in OpenAI compatibility is all you want
# Point any OpenAI client at:
#   base_url = http://localhost:11434/v1
#   api_key = "ollama"   (any non-empty value)

# OpenAI Python SDK
python3 - <<'PY'
from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
stream = client.chat.completions.create(
    model="qwen2.5:7b",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)
PY

Why you'd build this proxy

The shape of the conversion

Don't reinvent OpenAI compatibility

Code

External links

Exercise

Progress

Comments 0