Tool Calling Best Practices

Hard-won rules

Validate every tool call. Models hallucinate function names, miss required args, supply wrong types. Wrap dispatch in try/except and return error JSON to the model rather than crashing.
Keep the tool set small. 5–10 tools is a sweet spot. Beyond that, the model gets distracted and picks wrong. If you need more, group tools by domain and pick a subset per request.
Write descriptions like docstrings. First sentence: what it does. Second sentence: when to use it (and when not to).
Test with stream: false first. Streaming + tool calls adds parsing complexity. Get the loop right non-streaming, then layer streaming on top.
Use a registry. A {name: callable} dict beats a giant if/else chain — easier to test, easier to extend, easier to replace one tool.

The Ollama Python library shortcut

Since v0.4 the official library accepts Python functions directly as tools. It auto-generates the JSON Schema from the function signature and docstring. For prototyping, this is a huge accelerator. For production, define the schema explicitly so descriptions are intentional.

Code

Defensive dispatch with timeout·python

import asyncio, json, signal
from contextlib import contextmanager

class ToolError(Exception): ...

@contextmanager
def timeout(seconds: int):
    """POSIX SIGALRM-based timeout (single-thread only)."""
    def handler(_signum, _frame):
        raise TimeoutError(f"tool exceeded {seconds}s")
    old = signal.signal(signal.SIGALRM, handler)
    signal.alarm(seconds)
    try:
        yield
    finally:
        signal.alarm(0)
        signal.signal(signal.SIGALRM, old)

def dispatch(name: str, args: dict, registry: dict, timeout_s: int = 10) -> str:
    if name not in registry:
        return json.dumps({"error": f"Unknown tool: {name}"})
    try:
        with timeout(timeout_s):
            result = registry[name](**args)
        return result if isinstance(result, str) else json.dumps(result)
    except TypeError as e:
        return json.dumps({"error": f"Bad arguments: {e}"})
    except Exception as e:
        return json.dumps({"error": str(e)})

Ollama Python library — function-as-tool·python

from ollama import chat

def get_temperature(city: str) -> str:
    """Get the current temperature for a city. Returns string like '22C in Seoul'."""
    return f"22C in {city}"

# The library generates the JSON Schema from the docstring + signature
response = chat(
    model="qwen2.5:7b",
    messages=[{"role": "user", "content": "Temperature in Paris?"}],
    tools=[get_temperature],
)
print(response.message.content)

Tool Calling Best Practices

Hard-won rules

The Ollama Python library shortcut

Code

External links

Exercise

Progress

Comments 0