Options & Structured Outputs

The options block — fine control over generation

Every generation parameter lives under the options object. The most common ones:

temperature — randomness (0 = deterministic, 1.5+ = chaotic). Default 0.8.
top_p — nucleus sampling cutoff. Default 0.9.
top_k — top-K sampling. Default 40.
num_ctx — context window in tokens. Defaults vary by model (often 4096; bump to 8192 / 16384 / 32768 for real use).
num_predict — max tokens to generate. -1 = unlimited.
repeat_penalty — penalty for repeated tokens. Default 1.1.
seed — integer; fix for reproducible runs.
stop — array of stop sequences.
num_gpu — GPU layers. 999 = all on GPU.

Structured outputs via JSON Schema

The format field accepts a JSON Schema object. The model is forced to emit JSON matching that schema. This is one of the most useful features in the API — it turns a chat model into a typed data extractor without prompt-engineering for JSON.

When to use options vs Modelfile

Options on the request override Modelfile defaults. Bake stable defaults into the Modelfile (this variant is the coder; ctx is always 16K), but vary temperature, seed, and num_predict per request.

Code

Tuned generation with options·python

import httpx, json

resp = httpx.post(
    "http://localhost:11434/api/chat",
    json={
        "model": "qwen2.5:7b",
        "messages": [{"role": "user", "content": "Write three creative haiku about the sea."}],
        "stream": False,
        "options": {
            "temperature": 0.95,    # creative
            "top_p": 0.95,
            "num_ctx": 8192,
            "num_predict": 300,
            "seed": 42,             # reproducible
            "stop": ["---"],
        },
    },
    timeout=120.0,
)
print(resp.json()["message"]["content"])

Structured output with JSON Schema·python

schema = {
    "type": "object",
    "properties": {
        "subject": {"type": "string"},
        "verb": {"type": "string"},
        "adjectives": {"type": "array", "items": {"type": "string"}},
        "word_count": {"type": "integer"},
    },
    "required": ["subject", "verb", "adjectives", "word_count"],
}

resp = httpx.post(
    "http://localhost:11434/api/chat",
    json={
        "model": "qwen2.5:7b",
        "messages": [{
            "role": "user",
            "content": "Analyze: 'The quick brown fox jumps over the lazy dog'",
        }],
        "stream": False,
        "format": schema,
    },
    timeout=120.0,
)
result = json.loads(resp.json()["message"]["content"])
print(result)
# {'subject': 'fox', 'verb': 'jumps', 'adjectives': ['quick', 'brown', 'lazy'], 'word_count': 9}

Options & Structured Outputs

The options block — fine control over generation

Structured outputs via JSON Schema

When to use options vs Modelfile

Code

External links

Exercise

Progress

Comments 0