This is the second-most-common Ollama footgun (after NDJSON-vs-SSE). Ollama wants images as raw base64 strings in an images array, attached to the message. NO data:image/png;base64, prefix. NO content-array shape with image_url objects.
Copy the OpenAI shape into Ollama's native /api/chat and you get a loud HTTP 400, not a quietly text-only answer: json: cannot unmarshal array into Go struct field Alias.content of type string. Ollama declares Message.Content as a plain string, so a content array cannot be decoded at all. The data: prefix fails just as loudly — images entries decode as raw base64 bytes, so data:image/png;base64,… returns illegal base64 data at input byte 4.
The two endpoints want opposite shapes, and each rejects the other's:
Endpoint
Wants
Given the other shape
/api/chat (native)
images: ["<bare base64>"]
400 — cannot unmarshal array into … content of type string
/v1/chat/completions (OpenAI-compatible)
content array with image_url.url = "data:image/png;base64,…"
400 — invalid image input for a bare base64 string
That is the good news in this footgun: whichever way you get it wrong, the error body names the field. Read it instead of guessing.
Resize before sending
Local vision models often have a max input dimension (~896 to ~1568 px depending on the model). Resize to ~1024 px on the long edge before encoding. Larger images either get downscaled internally (wasting your encode) or hit context limits.
Code
Send a single image to a vision model·python
import httpx, base64
from pathlib import Path
def encode_image(path: str | Path) -> str:
"""Read a file and return its base64 string (no data URI prefix)."""
return base64.b64encode(Path(path).read_bytes()).decode("utf-8")
resp = httpx.post(
"http://localhost:11434/api/chat",
json={
"model": "gemma3:12b",
"messages": [{
"role": "user",
"content": "Describe what you see in this image. Be specific about objects and any text.",
"images": [encode_image("screenshot.png")],
}],
"stream": False,
},
timeout=300.0, # vision models are slower
)
print(resp.json()["message"]["content"])
Resize first to keep things fast·python
from PIL import Image
import io, base64
def encode_resized(path: str, max_edge: int = 1024) -> str:
img = Image.open(path)
if img.mode != "RGB":
img = img.convert("RGB")
img.thumbnail((max_edge, max_edge), Image.LANCZOS)
buf = io.BytesIO()
img.save(buf, format="JPEG", quality=88)
return base64.b64encode(buf.getvalue()).decode("utf-8")
# 1024px JPEG q88 keeps quality high but cuts encode + transfer + decode time roughly 4x for typical phone photos.
Encode a real screenshot from your computer at 1024px max edge. Send it to Gemma 3 12B with a question that requires actually looking at the image (e.g. 'What's the tab title in the browser?'). If the request 400s, read the error body — it names the field. If it returns 200 and the answer is still generic, the image decoded but did not help: check the model's vision capability and how far you downscaled.
Progress
Progress is local-only — sign in to sync across devices.