What Are Agents

1. TL;DR

An AI agent is an LLM placed inside a loop, given tools and memory, and pointed at a goal — the model itself decides, at runtime, what step to take next. It matters because the single hardest architecture decision in applied AI is "should the LLM drive, or should my code drive?" — get it wrong and you either burn tokens on a task a fixed pipeline would nail, or you hard-code a brittle flow where the task genuinely needs runtime judgment.

2. Simple explanation

Three things get confused constantly. Let's separate them cleanly.

• Plain LLM call: one prompt in, one completion out. No tools, no loop, no state. "Summarize this email." It's a function.
• Workflow: an LLM (or several) wired together through code paths you wrote. You decide the order of steps. The LLM fills in the blanks at each step, but your if/else and your graph edges decide what happens next. Predictable, testable, cheap.
• Agent: you hand the LLM a goal and a set of tools, and the LLM decides which tool to call, in what order, and when it's done. The control flow lives in the model's head, not in your source file.

The analogy that sticks: A workflow is a GPS route — turn-by-turn directions computed ahead of time; the car just follows them. An agent is a driver who knows the destination, can read road signs (observe), and chooses each turn as conditions change (roadblock? detour). The GPS is reliable and you can print it out. The driver handles the unexpected but might get lost.

The one test — memorize this for interviews:

At runtime, who decides the next step — my code, or the LLM? Code decides ⇒ workflow. LLM decides ⇒ agent.

This is exactly Anthropic's distinction in Building Effective Agents: workflows are systems where LLMs and tools are orchestrated through predefined code paths; agents are systems where LLMs dynamically direct their own processes and tool usage. Google's Agent Development Kit (ADK) draws the same line: Workflow Agents (Sequential / Parallel / Loop) execute sub-agents in a developer-defined order, while LLM Agents use the model to decide routing and which tools to call.

3. Diagram

PLAIN LLM CALL
  prompt ──► [ LLM ] ──► output                 (one shot, stateless)

WORKFLOW  (YOUR code decides the path)
  input ─► [step A] ─► [step B] ─► if X ─► [step C] ─► out
                 ▲            the arrows are edges YOU wrote
                 │            LLM fills each box; it does NOT pick the next box

AGENT  (the LLM decides the path)
             ┌─────────────────────────────────────────┐
  goal ─────►│  ┌────────┐  think  ┌──────────────┐     │
             │  │  LLM   │────────►│ pick a tool? │     │
             │  │ (brain)│         └──────┬───────┘     │
             │  └───▲────┘                │ tool call   │
             │      │ observation   ┌─────▼──────┐      │
             │      └───────────────│ run tool   │      │
             │        (loop)        └────────────┘      │
             │  memory ◄── read/write across the loop   │
             └──────────────────┬──────────────────────┘
                                │ LLM decides "done"
                                ▼
                              answer

4. How it works

An agent is not a model — it's a runtime pattern built from four ingredients:

Agent = LLM + Tools + Loop + Memory.

1. LLM (the reasoning core). A chat/instruct model that can emit either a final answer or a request to use a tool. Modern models are post-trained for tool calling: given tool schemas, they output a structured call instead of prose.
2. Tools. Typed functions the model may invoke — web search, a database query, a calculator, another agent. Each has a name, a description, and an argument schema. Tools are how the agent acts on the world and reads fresh state.
3. The loop (the "agentic" part). The engine that turns one model call into autonomy:

   while not done:
       action = LLM(goal, history, tool_schemas)   # model decides
       if action is a tool call:
           result = run_tool(action)                # your code executes
           history.append(result)                   # feed observation back
       else:
           done = True                              # model emitted final answer
   

   The crucial line is action = LLM(...): the next step is a model output, not a branch in your code. That single fact is what makes it an agent.
4. Memory. Short-term (the running message/scratchpad history in this loop) and long-term (persisted facts, past sessions, a vector store). Without memory the loop can't accumulate progress toward the goal.

Why "loop" is load-bearing: a plain LLM call and a single tool call are both one-shot. Agency emerges when the model can observe the result of its own action and decide what to do next — possibly many times. Termination is itself a model decision (the model stops asking for tools), which is powerful and also why agents can loop forever if you don't cap iterations.

The spectrum, not a binary. Real systems mix both. Anthropic's guidance is explicit: start with the simplest thing that works, and add agency only when the flexibility is worth the cost. Most production systems are workflows with one small agentic pocket, not fully autonomous agents. The augmented-LLM (LLM + retrieval + tools) is the shared building block underneath both.

5. The math

Agents don't have a single governing equation, but the loop has a useful formalism — it's a partially observable decision process the LLM drives:

At step t the policy (the LLM) chooses an action:

aₜ  ~  π_LLM( · | g, h₋ₜ )

• aₜ — the action at step t (a tool call, or "finish").
• π_LLM — the policy, i.e. the LLM conditioned on its prompt. In a workflow this policy is constrained to a fixed template; in an agent it's free over the whole tool set.
• g — the goal (system prompt + task).
• h₋ₜ — the history so far: h₋ₜ = (a₀, o₀, …, a_{t−1}, o_{t−1}), where oᵢ is the observation (tool result) after action aᵢ.

The observation update after acting:

oₜ = ENV(aₜ)          # your code runs the tool, returns a result
hₜ = h₋ₜ ⊕ (aₜ, oₜ)   # append action+observation to history (⊕ = concat)

Expected cost is why the workflow-vs-agent choice is economic, not just architectural. If each step costs c tokens and the loop runs N steps:

E[cost] = c · E[N]

For a workflow, N is fixed and known (you wrote the graph). For an agent, N is a random variable the model controls — bounded only by your max_iterations cap. Unbounded E[N] is the source of runaway agent bills and latency.

6. Real code

Below is a chat chain from a single-agent trip-planning workflow — a workflow, written in LangChain LCEL. Notice that the Python code dictates the order: parse → maybe-rewrite → route → retrieve → recommend. The LLM fills each box; it never chooses the next box.

# trip_planner.py  (example, abridged)
async def invoke(self, query, chat_id, user_profile=None):
    history_str = format_history_str(chat_id)

    # Step 1: parse intent (LLM call, structured output)
    parsed = await self._parse_intent(query, history_str)

    # Step 2: YOUR code decides whether to rewrite — not the model
    search_query = parsed.cleaned_query
    if parsed.is_relative and history_str:
        search_query = await self._rewrite_followup(parsed.cleaned_query, history_str)

    # Step 3: YOUR if-statement routes greetings away from retrieval
    if parsed.intent in ("greeting", "off_topic"):
        text = await asyncio.to_thread(self._greeting_chain.invoke, {"query": search_query})
        add_exchange(chat_id, query, text)
        return {"response": {"text": text, "destinations": []}, "chat_id": chat_id}

    # Step 4: YOUR if-statement asks for a location when missing
    if not effective_location:
        ...

    # Step 5: retrieve, then Step 6: recommend — fixed order, always
    docs = await asyncio.to_thread(retriever.invoke, search_query)
    text = await asyncio.to_thread(self._recommend_chain.invoke, {...})
    add_exchange(chat_id, query, text)   # Step 7: persist to memory

The control flow (if parsed.intent in (...), the fixed step order) is hand-written. That's the signature of a workflow.

Now here's what the same task would look like as a true agent — you hand the LLM the goal and the tools and let it decide the sequence:

# The agentic version: LLM picks the tools and the order, in a loop.
from anthropic import Anthropic
client = Anthropic()

TOOLS = [
    {"name": "parse_preferences", "description": "Extract budget, dates, destination from a message",
     "input_schema": {"type": "object", "properties": {"message": {"type": "string"}},
                      "required": ["message"]}},
    {"name": "search_flights", "description": "Find flights matching a travel query for a destination",
     "input_schema": {"type": "object",
                      "properties": {"query": {"type": "string"}, "location": {"type": "string"}},
                      "required": ["query"]}},
]

def run_agent(goal: str, max_iterations: int = 6):
    messages = [{"role": "user", "content": goal}]
    for _ in range(max_iterations):                       # the CAP that bounds E[N]
        resp = client.messages.create(
            model="claude-sonnet-4-5", max_tokens=1024,
            system="You are a trip-planning agent. Use tools to reach the goal, "
                   "then answer the user.",
            tools=TOOLS, messages=messages,
        )
        if resp.stop_reason != "tool_use":                # THE MODEL decided it is done
            return resp.content[0].text
        messages.append({"role": "assistant", "content": resp.content})
        results = []
        for block in resp.content:
            if block.type == "tool_use":
                out = dispatch(block.name, block.input)   # YOUR code executes the tool
                results.append({"type": "tool_result", "tool_use_id": block.id, "content": out})
        messages.append({"role": "user", "content": results})  # feed observations back
    return "Stopped: hit iteration cap."

Same trip-planning task. The difference is not the tools — it's who sequences them. In trip_planner.py your ifs do it; in run_agent the resp.stop_reason == "tool_use" branch means the model is choosing the next step every iteration.

7. Real-world example

A single-agent trip-planning workflow is the cleanest illustration because it can contain both patterns in one codebase:

• The solo chat path (trip_planner.py, shown above) is a pure workflow: a LangChain LCEL pipeline QueryParser → FollowupRewrite → DestinationRetriever → Recommender → Memory. Latency and cost are predictable; it's trivially unit-testable; and for a bounded task ("recommend a destination from a message") that's exactly what you want. No agency needed.
• The group-trip path (group_trip_graph.py) is a LangGraph state machine — still a workflow, but a branchier one: classify_intent → classify_request → {answer_followup | retrieve → suggest | ask_task_confirmation | create_task}. The LLM classifies at each node, but the edges are developer-defined (add_conditional_edges with an explicit routing map). Runtime who-decides-next? Still your code — the routing function reads a state field and returns a fixed node name. So: workflow.

Contrast a trip-planning assistant, which is agentic: an orchestrator LLM is handed four specialist agents as tools and decides at runtime which specialists to call for a given query — "a 5-day trip under $2000 with good weather" might fan out to flights + hotels + activities + budget, while "cheapest weekend getaway nearby" calls only two. Nobody wrote that branching; Claude 3.5 Sonnet chose it. That's the workflow→agent line, drawn with the same trip-planning task.

8. Interview questions companies actually ask

Q1. What is the difference between a workflow and an agent? [medium] Workflows orchestrate LLMs and tools through predefined code paths you wrote; agents let the LLM dynamically direct its own process and tool use. The one-line test: at runtime, who decides the next step — your code or the model? This is Anthropic's exact framing in Building Effective Agents. (Anthropic)

Q2. What are the minimal components of an agent? [easy] An LLM (reasoning core), tools (to act and observe), a loop (observe→decide→act until done), and memory (short-term scratchpad + optional long-term store). Drop the loop and you have a single tool call; drop the tools and you have a chatbot.

Q3. When would you NOT build an agent? [medium] When the task is well-defined and the steps are known ahead of time — a workflow gives you predictability, lower latency, lower cost, and testability. Anthropic's own advice: find the simplest solution and only add agency when flexibility outweighs the cost. Most production systems need a tight workflow, not another autonomous agent. (Anthropic, Mervin Praison)

Q4. How is an agent different from a plain chatbot? [easy] A chatbot is reactive: message in, message out. An agent is goal-directed and proactive — it plans, uses tools, does multi-step reasoning, and loops until the goal is met, deciding for itself when it's finished.

Q5. What makes the "loop" the defining feature of agency? [hard] Because agency = the ability to observe the result of your own action and choose the next action. A one-shot LLM call or a single tool call can't do that. The loop is what lets the model accumulate observations and self-terminate; termination itself becomes a model decision, which is both the power and the danger (infinite loops, runaway cost) — hence you always cap max_iterations. (The AI Engineer)

Q6. Anthropic and Google both split "workflow" from "agent" — reconcile them. [medium] Same line, different vocabulary. Anthropic: workflows = predefined code paths, agents = LLM directs itself. Google ADK: Workflow Agents (Sequential/Parallel/Loop) run sub-agents in a developer-defined order; LLM Agents use the model for routing and tool choice. Both anchor on who controls the flow at runtime. (AgentPatterns)

Q7. Give an example of the same task built both ways. [medium] Trip planning. Workflow: parse → retrieve → recommend, order fixed in code (the LCEL chain above). Agent: hand the model parse and search tools plus the goal and let it sequence them (the orchestrator). Identical tools; the difference is who orders them.

Q8. Why do agents cost and latency vary so much more than workflows? [hard] In a workflow the number of steps N is fixed, so E[cost] = c·N is known. In an agent N is a random variable the model controls; without a hard cap, expected cost is unbounded and tail latency explodes. You bound it with max_iterations, step budgets, and cheaper models for inner steps.

Q9. Is a routing/classification step enough to call something an agent? [hard] No. Routing where your code reads the classification and jumps to a fixed node (like the LangGraph add_conditional_edges above) is still a workflow — the model informs the branch but doesn't own the control flow. It becomes an agent only when the model itself chooses which tool/step comes next with no predefined map.

9. When to use / tradeoffs

Rules of thumb:

• Default to the simplest thing. Plain call → workflow → agent, in that order of preference.
• Reach for an agent only when the set/order of steps genuinely can't be known ahead of time (task structure emerges at runtime) and the flexibility is worth the cost, latency, and error-compounding risk.
• Even in agentic systems, wrap the agent in a workflow skeleton (validation, guardrails, retries) so the autonomy is contained.
• Always cap iterations and log the full action/observation trace — agents are only as debuggable as their traces.

10. Summary + related articles

An agent is LLM + tools + loop + memory, and the line separating it from a workflow is a single runtime question: who decides the next step — your code (workflow) or the model (agent)? Anthropic and Google ADK draw the same line. Prefer the simplest design that works; add agency only when the task's shape is genuinely unknown until runtime, and always bound the loop.

Related articles:

• Agent Architectures — the four layers of an agent and single- vs multi-agent designs.
• Reasoning Patterns — ReAct, Reflexion, Plan-and-Execute, ReWOO, ToT.
• Tool Use — how tool/function calling actually works, end to end.

Sources: Anthropic — Building Effective Agents · When Not to Build AI Agents (Anthropic's playbook) · AgentPatterns — Effective Agents Framework · The AI Engineer — Single-Agent Patterns