Before Tool Calling — The Workarounds

Long before tool calling was a built-in feature of any model, people were already trying to break the LLM out of its room. The early workarounds are worth knowing because they are still alive in older code, and because they explain why today's clean APIs feel like such a relief.

The first family of approaches was prompt-and-parse: ask the model to produce a structured string — JSON, XML, a special <action> tag — and run a regex over its output to detect "I want to call this function." The reliability story was grim. The model would forget the format mid-sentence, wrap the JSON in chatty preamble, switch quote styles, or hallucinate fields. Every team built their own retry-and-repair loop, and every team's loop leaked corner cases.

The second family was ReAct-style scaffolding: prompt the model to interleave Thought / Action / Observation lines, parse the Actions out of the stream, execute them, then feed the Observations back in. ReAct was a real intellectual leap — it taught us that reasoning and acting are interleaved, not sequential — but as an engineering substrate it was paper-thin. The model still had to perfectly format every step, and the parser still had to be defensive against poetry.

The third family was full agent frameworks: LangChain agents, AutoGPT, BabyAGI. They wrapped the prompt-and-parse loop in object-oriented machinery and added planners, memory, and self-critique. They worked when they worked. They failed in spectacular, hard-to-debug ways when they did not, because every layer was still building on the same brittle "ask the model to please format itself" foundation.

The lesson hidden in all of this is the lesson of every protocol we are about to study: the contract has to live below the prose. As long as tool invocation is a thing the model is asked to type out as part of its English answer, you are gambling on the model's mood. The breakthrough was making tool invocation a first-class output of the API itself — not text the model writes, but a structured field the inference engine emits.

Code

ReAct-flavored prompt-and-parse (the old pain)·python

# An LLM is asked to produce a strict format. A regex extracts the call.
# Real systems had to handle: extra whitespace, alternate JSON quoting,
# leaked thoughts, wrapped code fences, mid-stream apologies, etc.
import re, json

prompt = '''You are an assistant with one tool: get_weather(location: str).
When you need it, output exactly:
ACTION: {"tool": "get_weather", "args": {"location": "<city>"}}
Otherwise reply normally.'''

raw = call_model(prompt + "\nUser: weather in Seoul?")
# raw might be:
#   "Sure! ACTION: {\"tool\": \"get_weather\", \"args\": {\"location\":\"Seoul\"}}"
m = re.search(r"ACTION:\s*({.*})", raw)
if m:
    try:
        call = json.loads(m.group(1))
    except json.JSONDecodeError:
        call = repair_json(m.group(1))  # every team had to write this

Before Tool Calling — The Workarounds

Code

External links

Exercise

Progress

Comments 0