Tool Choice and the Agentic Loop

Once a tool list is on the table, the next decision is tool choice: how much freedom does the model have on this turn? Every major provider exposes the same three modes under different names.

auto (default): the model chooses whether to call a tool or answer in text. Use this for normal conversational agents.
any / required: the model must call some tool — it is not allowed to answer in plain text. Use this when the architecture requires a tool result for the next step (e.g. a router that always emits a routing decision).
specific tool: force the call to a named tool. Useful for the first turn of a structured workflow ("always start by calling plan_steps").

Tool choice is the pressure valve for the agentic loop. The most common bug — agents that wax philosophical without ever doing anything — is fixed by switching the first turn to any or to a specific tool. The other common bug — agents that loop forever calling the same tool — is fixed by leaving choice on auto after the first turn so the model can choose to stop and answer.

The agentic loop wraps these decisions with structure. A robust loop has three pieces of mechanical safety: a max_turns cap (so a buggy tool description does not produce a permanent loop), an error feedback path (when a tool throws, return the error to the model as a tool result so it can adjust), and a cost guard for production (stop and ask when total tokens or wall time crosses a threshold). These are not premature optimizations; they are the difference between an agent that recovers and one that crashes silently.

Code

Tool choice — three providers, same idea·python

# Anthropic
client.messages.create(..., tool_choice={"type": "any"})
client.messages.create(..., tool_choice={"type": "tool", "name": "plan_steps"})

# OpenAI Responses
client.responses.create(..., tool_choice="required")
client.responses.create(..., tool_choice={"type": "function", "name": "plan_steps"})

# Gemini
GenerationConfig(tool_config=ToolConfig(function_calling_config=FunctionCallingConfig(mode="ANY")))

Loop with error feedback·python

def call_tool_safely(name, args, executors):
    try:
        return {"ok": True, "result": executors[name](**args)}
    except Exception as e:
        return {"ok": False, "error": f"{type(e).__name__}: {e}"}

# Inside the loop, send error results back as tool_result with the error
# message in 'content' and let the model decide whether to retry, switch
# tools, or apologize to the user.

Exercise

Build a two-tool agent: a 'lookup' tool that always succeeds and a 'broken' tool that always raises. Run a prompt that selects the broken tool. Verify that returning the error as a tool_result lets the model recover by selecting 'lookup' instead, instead of the loop crashing. That recovery path is the test for whether your loop is honest about failure.

Tool Choice and the Agentic Loop

Code

External links

Exercise

Progress

Comments 0