Tool Use

1. TL;DR

Tool use (a.k.a. function calling) is the mechanism that lets an LLM act: you hand the model tool schemas, it emits a structured call (tool name + typed arguments), your code runs it, you feed the result back, and the loop continues. It matters because tools are what turn a text predictor into an agent that can search, query databases, do math, and touch the real world — and doing it reliably (validated args, sandboxing, error handling) is the difference between a demo and production.

2. Simple explanation

An LLM alone can only produce text. It can't look up today's flight schedule, run a price calculation, or query your database — it just predicts tokens. Tool calling fixes that by giving the model a set of functions it's allowed to request.

The key mental model: the LLM never runs anything. It only fills out an order form. You describe each tool ("search_flights(query, location) — finds flights"), and when the model wants to use one, it returns a structured object: "call search_flights with {query: 'nonstop to Lisbon', location: 'Boston'}." Your code reads that order, actually executes the function, and hands the result back to the model, which then decides what to do next.

Analogy: the LLM is a smart manager who can't leave the office. It writes work orders ("get me the Q3 numbers from the database") and passes them to staff (your code). The staff run the query and bring back results. The manager reads them and writes the next order — or the final report. Structured output is the strict form the work order must be filled in on so the staff can act without guessing.

3. Diagram

                    THE TOOL-CALLING LOOP
  ┌─────────────────────────────────────────────────────────────┐
  │  1. You send: prompt + TOOL SCHEMAS (name, desc, arg types)  │
  │                        │                                      │
  │                        ▼                                      │
  │                    ┌───────┐                                  │
  │                    │  LLM  │  decides: answer, or call a tool?│
  │                    └───┬───┘                                  │
  │            tool_use    │    text                              │
  │        ┌───────────────┴──────────────┐                       │
  │        ▼                              ▼                        │
  │  2. {name:"search",            5. FINAL ANSWER ──► done        │
  │      args:{q:"lisbon"}}                                        │
  │        │  (structured call, NOT prose)                        │
  │        ▼                                                       │
  │  3. VALIDATE args ─► SANDBOX ─► run_tool() ─► try/except       │
  │        │  (YOUR code executes — the model never does)         │
  │        ▼                                                       │
  │  4. tool_result ──► append to messages ──┐                    │
  │        └───────────── loop back to LLM ◄──┘                    │
  └─────────────────────────────────────────────────────────────┘

4. How it works

Step 1 — Advertise tools as schemas. Each tool is described with a name, a natural-language description (the model routes on this — write it well), and a JSON-Schema for its arguments (types, required fields, enums). This schema is injected into the request. Modern models are post-trained specifically to consume these and emit conforming calls.

Step 2 — The model emits a structured call. Instead of prose, the model returns a tool_use block: the chosen tool's name and a JSON object of arguments. Crucially the API returns a stop_reason: "tool_use" signal so your loop knows it's a call, not a final answer. The model can request several tools at once (parallel tool calls).

Step 3 — Your code executes. You look up the function, validate the arguments against the schema (never trust raw model output), run it — ideally sandboxed and with a timeout — and capture the result or the error.

Step 4 — Feed the result back. You append a tool_result message (tagged with the call's ID) to the conversation and call the model again. Now it can reason over the observation.

Step 5 — Loop until done. Repeat 2–4 until the model returns text with a normal stop reason (no more tool calls) — that's the final answer. This is exactly the ReAct loop; tool use is the "Act/Observe" half.

Two flavors of structured output — know the distinction:

• Tool/function calling: the model chooses whether to call, and may respond with plain text instead. Good for agents that sometimes act, sometimes answer.
• Structured output / JSON mode: the model is forced to fill a schema every time (e.g. always return a ParsedQuery object). Good for extraction/parsing where you always want typed data. In Python, Pydantic is the standard: define a BaseModel, and a parser coerces the model's output into it. A PydanticOutputParser does exactly this.

Reliability — the part that separates seniors from juniors:

1. Validate arguments — the model can hallucinate fields, wrong types, or out-of-range values. Parse against the schema (Pydantic/JSON-Schema); reject or repair before executing.
2. Prefer API-native structured modes / constrained decoding — "strict mode," JSON mode, or grammar-constrained decoding enforce valid JSON at generation time, far more reliable than regex-scraping prose. Schema validation at the API level beats text parsing.
3. Sandbox & least privilege — run tools with timeouts, in isolated environments, with scoped credentials. A tool that can rm -rf or spend money needs guardrails and sometimes human approval.
4. Handle errors gracefully — wrap every tool in try/except and return the error as an observation so the model can recover or retry, rather than crashing the loop.
5. Guard the whole surface — rate limits, output moderation/guardrails, and idempotency for side-effecting tools.

5. The math

Tool use isn't equation-heavy, but two relationships come up in interviews.

Tool-routing accuracy vs number of tools. The model must pick the right tool among K; distractors hurt:

P(correct call)  ≈  P_name(right tool | K)  ·  P_args(valid args | right tool)

• P_name — probability of selecting the correct tool; degrades as K (tool count) grows and as descriptions overlap. Keep tool sets small and descriptions distinct (this is why multi-agent decomposition helps).
• P_args — probability the arguments validate; improved by strict schemas, enums over free text, and constrained decoding.

Expected calls before a valid tool call, if each attempt validates with probability p and you allow repair-retries:

E[attempts] = 1 / p

• p — per-attempt argument-validity rate. Pushing p→1 with strict/JSON mode directly cuts wasted round-trips and latency. This is the quantitative case for constrained decoding.

6. Real code

A budget tool — the @tool decorator turns a plain Python function into a schema-advertised, agent-callable tool. The docstring and type hints become the schema the model reads:

# budget_tools.py  (example)
from strands import tool

@tool
def calculate_budget(max_price: float, price_per_person: bool = True) -> str:
    """Calculate budget filter for flight search.

    Returns price scale: 1=$, 2=$$, 3=$$$, 4=$$$$
    """                                     # ^ description + typed args = the schema
    if max_price <= 10:   result, level = "1", "$"
    elif max_price <= 30: result, level = "1,2", "$-$$"
    elif max_price <= 60: result, level = "2,3", "$$-$$$"
    else:                 result, level = "3,4", "$$$-$$$$"
    print(f"✅ [Budget Filter] Result: {result} ({level})")
    return result                           # the tool_result fed back to the model

A tool with error handling — note it returns the error as a string observation so the agent can recover instead of crashing:

# flight_tools.py  (example, abridged)
@tool
def search_flights(query: str, location: str = None) -> str:
    """Search for flights using a flight search API.

    Args:
        query: Search query (e.g., "nonstop flights to Lisbon")
        location: Location string (e.g., "Boston, MA") - will be added to query
    Returns:
        JSON string with flight data
    """
    try:
        client = get_flight_client()
        if client is None:
            return json.dumps({"error": "Flight API client not initialized"})  # graceful
        if location and location.lower() not in query.lower():
            query = f"{query} in {location}"
        return json.dumps(client.ai_chat_search(query=query), indent=2)
    except Exception as e:
        return f"Error: {str(e)}"           # error becomes an OBSERVATION, loop survives

Forced structured output — the forced-schema flavor, using LangChain's PydanticOutputParser. The rest of the pipeline gets typed data, not strings:

# query_parser.py  (example)
from langchain_core.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field

class ParsedQuery(BaseModel):
    intent: str = Field(description="greeting, trip_query, location_query, off_topic, follow_up")
    destination: Optional[str] = Field(None, description="Specific destination/city/region mentioned")
    location: Optional[str] = Field(None, description="City, neighbourhood, or area mentioned")
    trip_type: Optional[str] = Field(None, description="beach, city, adventure, or None")
    preferences: List[str] = Field(default_factory=list, description="Preferences to honor")
    is_relative: bool = Field(False, description="references something from prior conversation")
    cleaned_query: str = Field(description="Normalised, standalone version of the query")

_parser = PydanticOutputParser(pydantic_object=ParsedQuery)

def get_query_parser_chain():
    llm = ChatGroq(model=GROQ_MODEL, temperature=0, api_key=GROQ_API_KEY)
    return (
        _PROMPT.partial(format_instructions=_parser.get_format_instructions())  # schema → prompt
        | llm
        | _parser        # coerces the model's JSON into a validated ParsedQuery, or raises
    )

And it validates defensively — if parsing fails, it degrades instead of crashing (reliability rule #4 in code):

# trip_planner.py  (example)
async def _parse_intent(self, query, history_str):
    try:
        return await asyncio.to_thread(self._parser_chain.invoke,
                                       {"query": query, "history": history_str or "None"})
    except Exception as exc:
        print(f"[TRIP-PLANNER] Parser error ({exc}); using raw query")
        return ParsedQuery(intent="trip_query", cleaned_query=query, is_relative=False)  # fallback

A raw provider-level tool loop (no framework) to show the wire protocol:

from anthropic import Anthropic
client = Anthropic()
TOOLS = [{"name": "calculate_budget",
          "description": "Map a max price in USD to a price scale (1-4).",
          "input_schema": {"type": "object",
              "properties": {"max_price": {"type": "number"}},
              "required": ["max_price"]}}]

def run(user_msg):
    messages = [{"role": "user", "content": user_msg}]
    while True:
        r = client.messages.create(model="claude-sonnet-4-5", max_tokens=1024,
                                   tools=TOOLS, messages=messages)
        if r.stop_reason != "tool_use":                 # no tool wanted → final answer
            return r.content[0].text
        messages.append({"role": "assistant", "content": r.content})
        results = []
        for b in r.content:
            if b.type == "tool_use":
                try:
                    out = str(dispatch(b.name, b.input))       # validate + run (your code)
                except Exception as e:
                    out = f"ERROR: {e}"                         # error as observation
                results.append({"type": "tool_result", "tool_use_id": b.id, "content": out})
        messages.append({"role": "user", "content": results})  # feed back, loop

7. Real-world example

A trip-planning assistant is tool use end to end. Its four specialist agents are each just a prompt plus a small set of @tool functions — calculate_budget, search_flights, hotel/preference tools, activity tools. When the orchestrator delegates "trip under $2000" to the budget agent, Claude 3.5 Sonnet emits a structured call calculate_budget(max_price=2000, price_per_person=True); Strands validates the args against the decorated signature, runs the Python, and feeds "3,4" back so the agent can pass a price filter downstream. Every tool returns errors as strings, so a dead flight client degrades to a graceful message instead of a stack trace. A Bedrock guardrail wraps the whole surface — the "guard the whole surface" reliability rule.

A single-agent trip-planning workflow shows the other flavor — forced structured output. Its QueryParser doesn't "choose" to emit JSON; it must produce a ParsedQuery every time, so the downstream LCEL steps (follow-up rewrite, retrieval, recommendation) operate on typed fields (intent, location, preferences) instead of re-parsing prose. And _parse_intent wraps it in try/except with a sane fallback — validate, degrade, never crash.

A fixed multi-step review pipeline takes reliability furthest with its rule "LLM points, Python reads": the LLM's job is to point at evidence (emit a structured citation / selection), and deterministic Python does the actual reading, math (random-effects meta-analysis), and certainty rating. Tool use as a safety architecture — the model chooses, validated code executes.

8. Interview questions companies actually ask

Q1. Walk through the function-calling loop. [easy] Send prompt + tool schemas → model returns a structured tool_use call (name + args), signaled by stop_reason: "tool_use" → your code validates and runs it → you append a tool_result → call the model again → repeat until it returns a plain-text final answer. The model never executes anything. (ml4devs)

Q2. Structured output vs tool calling — what's the difference? [medium] Tool calling lets the model choose whether to call a function and may reply with text instead; structured output forces the model to fill a schema every response. Use tool calling for act-or-answer agents; use structured output for extraction/parsing where you always want typed data. (DEV, Towards Data Science)

Q3. How do you make tool calls reliable in production? [hard] Validate arguments against a schema (Pydantic/JSON-Schema) before executing; prefer API-native strict/JSON mode or constrained decoding over regex-scraping; sandbox with timeouts and least-privilege credentials; wrap every tool in try/except and return errors as observations; add guardrails/rate limits and human approval for dangerous side effects. (Agenta)

Q4. Why is a model's JSON output unreliable, and how do providers fix it? [hard] Free-form generation drifts: malformed syntax, wrong field names/types, and silent breakage after model updates. Providers fix it with constrained/strict decoding and post-training for JSON, enforcing the schema at generation time — schema validation at the API level beats fragile text parsing. (Agenta, Towards Data Science)

Q5. Where does the model "see" your tools — how does it know they exist? [medium] You inject tool schemas (name, description, argument JSON-Schema) into the request. The model was post-trained to consume these and emit conforming calls; it routes primarily on the description text, so clear, distinct descriptions materially improve tool selection.

Q6. A tool raises an exception mid-loop. What should happen? [medium] Catch it and return the error as a tool_result observation so the model can retry with different args or take another path — never let it crash the agent loop. The tools above literally return f"Error: {str(e)}".

Q7. How does tool count affect reliability, and what do you do about it? [hard] Selection accuracy drops as the number of tools grows and descriptions overlap (more distractors). Mitigate by keeping tool sets small and descriptions distinct, grouping tools behind sub-agents (multi-agent decomposition), or dynamically filtering which tools are visible per turn.

Q8. What are parallel tool calls and why do they matter? [medium] Models can emit multiple tool_use blocks in one turn; you execute the independent ones concurrently and return all results together. It cuts latency for fan-out steps (the same idea ReWOO exploits) without changing the loop.

Q9. How do Pydantic parsers fit in? [easy] You define a BaseModel schema; the parser injects format instructions into the prompt and then coerces/validates the model's output into a typed object, raising on mismatch. A ParsedQuery model gives the rest of the pipeline typed fields instead of raw strings. (Agenta)

Q10. When would you deliberately NOT give an agent a tool? [medium] When the action is dangerous/irreversible without human sign-off, when a deterministic code path is safer ("LLM points, Python reads"), or when adding the tool bloats the tool set and hurts routing for everything else. Least privilege applies to tools too.

9. When to use / tradeoffs

Rules of thumb: validate before you execute; prefer schema-enforcing modes over prose parsing; keep tool sets small with sharp descriptions; sandbox and scope credentials; return errors as observations; gate dangerous tools behind approval. Reach for structured output when you always want typed data, tool calling when the model should decide whether to act.

10. Summary + related articles

Tool use is the agent's hands: advertise schemas, the model emits a structured call, your code validates and runs it, you feed the result back, and loop. Two flavors — free-choice tool calling vs forced structured output (Pydantic/JSON mode). Reliability is the real skill: validate args, prefer constrained decoding, sandbox, handle errors as observations, guard the surface — exactly what the @tool functions, the PydanticOutputParser, and the "LLM points, Python reads" pipeline above demonstrate.

Related articles:

• What Are Agents — tools are one of the four ingredients (LLM + tools + loop + memory).
• Reasoning Patterns — tool use is the "Act/Observe" half of ReAct.
• Agent Architectures — the tool-integration layer and agent-as-tool pattern.

Sources: The guide to structured outputs and function calling — Agenta · Structured Outputs vs Tool Calling — DEV · Structured Outputs with LLMs — Towards Data Science · LLM Function Calling and Tools — ml4devs