Local Tool Calling 101

What tool calling is

Tool calling (function calling) lets the model request execution of an external function instead of answering directly. The model returns a structured request: function name + arguments. Your code executes the function, sends the result back, and the model continues from there. This is what makes LLMs act instead of just talk.

Tools as JSON Schema, OpenAI-style

Ollama uses the OpenAI tool format for compatibility:

Each tool has type: "function" and a function object.
function has name, description, and parameters (a JSON Schema).
The description is what the model reads to decide whether to call the tool — write it like a careful one-line docstring.

Not all models support tools

Tool calling depends on the model being trained for it. Reliable families: Qwen 2.5+ / 3 / 3.5, Llama 3.1+, Mistral / Mixtral, Command R, GPT-OSS, Qwen3-Coder. Check capability with ollama show MODEL and look for tools in the capabilities line.

Code

Define a tool, see what the model returns·python

import httpx, json

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city. Returns temperature and condition.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "City name, e.g. 'Tokyo'"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
            },
            "required": ["city"],
        },
    },
}]

resp = httpx.post(
    "http://localhost:11434/api/chat",
    json={
        "model": "qwen2.5:7b",
        "messages": [{"role": "user", "content": "What's the weather in Tokyo right now?"}],
        "tools": tools,
        "stream": False,
    },
    timeout=120.0,
)

msg = resp.json()["message"]
print(json.dumps(msg, indent=2))
# Expected: empty content, tool_calls populated with get_weather(city="Tokyo")

Verify tool capability before sending·bash

# Don't waste tokens on a model that doesn't support tools
ollama show qwen2.5:7b | grep -i capabilities
# Look for: 'tools' in the list

# If you don't see 'tools', either upgrade the model or use a tools-capable variant

What tool calling is

Tools as JSON Schema, OpenAI-style

Not all models support tools

Code

External links

Exercise

Progress

Comments 0