POST /api/generate — Raw Completion · Local AI Quest

When to use /api/generate

/api/generate takes a single prompt string instead of a messages array. It's the right shape when:

You're doing fill-in-the-middle (FIM) code completion. The suffix field holds the code after the cursor; the model fills the gap.
You want a one-shot completion without conversation framing.
You need to send raw mode (raw: true) and bypass the model's chat template entirely — useful for fine-tuned models with custom prompt formats.

FIM is the killer use case

FIM is not a property of the endpoint — it is a property of the model's template. suffix only does anything if the model's Ollama template has a .Suffix branch that wraps your prefix and suffix in the model's own fill-in-the-middle tokens; otherwise the field is quietly ignored. The families that carry one today: qwen2.5-coder (every tag, base and instruct), deepseek-coder-v2, starcoder2, and Code Llama's code tags such as codellama:code. Check before you trust it: ollama show MODEL --template | grep -c Suffix, or look for insert in the model's Capabilities block. Given a prefix and a suffix, such a model completes the middle of a function. This is what powers local GitHub-Copilot-style autocomplete in Continue.dev, whose autocomplete role is explicitly FIM-based and lists Ollama as a provider with Qwen2.5-Coder 1.5B/7B as its open recommendations. Not every local-model tool uses this endpoint: Aider talks to Ollama through ollama_chat/, and Cursor's Tab model is its own hosted service, not your suffix field.

Other useful fields

images — array of base64 image strings (multimodal models).
options — same as /api/chat.
format — same as /api/chat.
raw — boolean; if true, no template is applied to the prompt.

EV v3 · 2026-09-18: The FIM example called qwen3-coder:7b, which has never been a tag — the library lists Qwen3-Coder at 30b and 480b only. The deeper error was the model family, not the size: Qwen3-Coder's Ollama template has no .Suffix branch at all, so even a real Qwen3-Coder tag would have ignored the suffix field and returned an ordinary completion. The example now uses qwen2.5-coder:7b, which does carry the FIM template, and the expected output in the comment is what it actually returned on ollama 0.34.1. The prose above now says what FIM depends on — the template, not the endpoint — and how to check it in one command, and it no longer credits Aider or Cursor with a suffix-based autocomplete neither of them does — Continue.dev is the one that documents this exact path. A silently-ignored suffix looks exactly like a model that is bad at filling holes.

Code

FIM completion with suffix·python

import httpx

# Code with a hole — the model fills the middle
prefix = '''def parse_csv(path: str) -> list[dict]:
    """Parse a CSV file and return rows as dicts."""
    with open(path, encoding="utf-8") as f:
        reader = csv.DictReader(f)
'''
suffix = '''
        return rows
'''

resp = httpx.post(
    "http://localhost:11434/api/generate",
    json={
        "model": "qwen2.5-coder:7b",
        "prompt": prefix,
        "suffix": suffix,
        "stream": False,
        "options": {"temperature": 0.2, "num_predict": 200},
    },
    timeout=60.0,
)
print(resp.json()["response"])
# measured on qwen2.5-coder:7b (ollama 0.34.1): '        rows = list(reader)'

Raw mode for custom templates·python

# When you have a fine-tuned model with its own format, bypass the chat template
custom_prompt = "<|im_start|>user\nHello<|im_end|>\n<|im_start|>assistant\n"

resp = httpx.post(
    "http://localhost:11434/api/generate",
    json={
        "model": "my-custom-model",
        "prompt": custom_prompt,
        "raw": True,           # skip Ollama's template
        "stream": False,
    },
)
print(resp.json()["response"])

Exercise

Implement a tiny complete(prefix, suffix, model) function using /api/generate with the suffix field. First confirm your model actually supports it — ollama show MODEL --template | grep -c Suffix should print 1, not 0. Use it to fill in the middle of three real Python snippets (a list comprehension, a try/except, a function body). Note when the suffix helps and when it confuses smaller models.

POST /api/generate — Raw Completion