Vision: Sending Images to Claude

~14 min · vision, images, multimodal

Level 0Observer

0 XP0/64 lessons0/13 achievements

0/150 XP to next level150 XP to go0% complete

Two ways to send an image

Image inputs go in as content blocks of type image. The block carries either base64 bytes (source: {type: 'base64', media_type: 'image/png', data: ...}) or a URL (source: {type: 'url', url: ...}). Use base64 for local files; URLs for assets already on a CDN.

Token cost of images

Images are billed by approximate token count derived from dimensions. Larger images cost more. Resize aggressively — a 1024-wide photo is usually plenty for OCR or object descriptions; 4K is wasted tokens. Anthropic's docs publish the conversion formula.

What vision is good at

Strong: chart and table OCR, diagram interpretation, scene description, UI screenshots, document layout, code in screenshots. Weaker: precise pixel coordinates, counting many similar items, fine-grained spatial reasoning. Frame your prompt to match the strengths.

Principle: Send the smallest image that still answers the question. Resize is a one-line cost win.

Code

Send a local PNG via base64·python

import base64, pathlib

data = base64.standard_b64encode(pathlib.Path("chart.png").read_bytes()).decode()

resp = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "image", "source": {
                    "type": "base64",
                    "media_type": "image/png",
                    "data": data,
                }},
                {"type": "text", "text": "Read off the y-axis values for the bars."},
            ],
        }
    ],
)

Resize before sending·python

from PIL import Image
import io, base64

img = Image.open("photo.jpg")
img.thumbnail((1024, 1024))  # bound to 1024 on the long side
buf = io.BytesIO()
img.save(buf, format="JPEG", quality=85)
data = base64.standard_b64encode(buf.getvalue()).decode()
# now send `data` as the image source

External links

Exercise

Take a vision feature in your project. Add a resize step bounded to 1024 on the long side. Compare token usage and answer quality on 10 sample inputs.

Hint

If quality drops noticeably at 1024, raise to 1536 or 2048 — but seldom higher. Most tasks do not care.

Progress

Progress is local-only — sign in to sync across devices.

← PreviousContext Windows: Long Is Not Free Next →PDF Input: Two Paths and When to Use Each

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.