Multiple Images and Use Cases

Multiple images per turn

The images array can hold several base64 strings; the model sees all of them in the same turn. This is how you do "compare these two charts" or "match this product photo to one of these reference photos" in a single round.

Real use cases that work locally

Screenshot analysis. "What's wrong with this UI?" — surprisingly good on Gemma 3 12B and up.
Document OCR. Qwen 2.5-VL is best in class for extracting structured data from receipts, invoices, forms.
Chart reading. Bar charts, line charts, simple heat maps — works well; complex multi-axis or stacked charts are still hit-or-miss.
Code-screenshot to code. Gemma 3 27B and Qwen 2.5-VL 32B can transcribe code from screenshots competently.
UI element identification. "Where's the submit button?" — useful for accessibility tooling and local UI agents.

What still struggles locally

High-resolution detail (text smaller than 30 px on the original).
Complex spatial reasoning ("Is the cat behind the chair or in front of it?").
Multi-page documents in one turn — break into per-page calls.
Hand-drawn diagrams with arrows / annotations.

Code

Compare two charts in one turn·python

resp = httpx.post(
    "http://localhost:11434/api/chat",
    json={
        "model": "qwen2.5vl:7b",
        "messages": [{
            "role": "user",
            "content": "Compare these two charts. Which shows higher peak revenue and in what year?",
            "images": [
                encode_resized("chart_2024.png"),
                encode_resized("chart_2025.png"),
            ],
        }],
        "stream": False,
    },
    timeout=600.0,
)
print(resp.json()["message"]["content"])

OCR a receipt to structured JSON·python

schema = {
    "type": "object",
    "properties": {
        "vendor": {"type": "string"},
        "date": {"type": "string"},
        "total": {"type": "number"},
        "currency": {"type": "string"},
        "line_items": {"type": "array", "items": {"type": "object",
            "properties": {"name": {"type": "string"}, "amount": {"type": "number"}},
            "required": ["name", "amount"],
        }},
    },
    "required": ["vendor", "date", "total", "currency", "line_items"],
}

resp = httpx.post(
    "http://localhost:11434/api/chat",
    json={
        "model": "qwen2.5vl:7b",
        "messages": [{
            "role": "user",
            "content": "Extract all fields from this receipt as JSON.",
            "images": [encode_resized("receipt.jpg")],
        }],
        "stream": False,
        "format": schema,
    },
    timeout=300.0,
)
import json
data = json.loads(resp.json()["message"]["content"])
print(json.dumps(data, indent=2, ensure_ascii=False))

Multiple Images and Use Cases

Multiple images per turn

Real use cases that work locally

What still struggles locally

Code

External links

Exercise

Progress

Comments 0