Agent Architectures

1. TL;DR

An agent architecture is the arrangement of four layers — reasoning, orchestration, memory, and tool-integration — plus the choice of how many agents you use and how they coordinate (single agent vs multi-agent; and among multi-agent, the dominant pattern is orchestrator-worker). It matters because the architecture, not the model, is what decides whether your system is debuggable, cheap, and correct at scale — interviewers probe this to see if you can tell when a second agent helps versus when it just adds latency and coordination bugs. Production reference architectures usually add two more first-class layers on top of the four — guardrails and observability — and that framing choice is itself a question you should be ready to answer.

2. Simple explanation

Think of an agent as a person doing a job:

• Reasoning layer = their brain — deciding what to do next.
• Orchestration layer = their calendar and to-do list — sequencing steps, handling retries, deciding when to stop.
• Memory layer = their notebook — what they've learned this session and across sessions.
• Tool-integration layer = their hands and phone — the APIs, databases, and functions they can actually operate.

A single-agent system is one skilled generalist doing the whole job alone. A multi-agent system is a team: a manager (orchestrator) who understands the request and delegates to specialists (workers) — a flights expert, a hotels expert, an activities planner — then stitches their answers into one recommendation. The manager doesn't book or pack; it coordinates. That "manager + specialists" shape is the orchestrator-worker pattern, and it's the multi-agent architecture you'll be asked about most.

3. Diagram

THE FOUR LAYERS (inside ONE agent)
┌───────────────────────────────────────────────┐
│  REASONING      LLM decides next action        │
│      ▲  │                                       │
│      │  ▼                                       │
│  ORCHESTRATION  loop · routing · retries · stop │
│      ▲  │                    ▲                  │
│      │  ▼                    │                  │
│  TOOL-INTEGRATION      MEMORY                   │
│  search · db · APIs    short-term + long-term   │
└───────────────────────────────────────────────┘

TEACHING FRAME (4)            PRODUCTION FRAME (4 + 2)
the four layers above          the same four, wrapped:

                        ┌─ GUARDRAILS ─────────────────────────┐
                        │  input guard · tool guard ·           │
                        │  schema validation · sandbox ·        │
                        │  fail-closed default                  │
                        │ ┌─ OBSERVABILITY ──────────────────┐ │
                        │ │  traces · cost + budget per task  │ │
                        │ │  run artifacts · replay            │ │
                        │ │ ┌──────────────────────────────┐  │ │
                        │ │ │  REASONING                   │  │ │
                        │ │ │  ORCHESTRATION               │  │ │
                        │ │ │  TOOL-INTEGRATION   MEMORY   │  │ │
                        │ │ └──────────────────────────────┘  │ │
                        │ └───────────────────────────────────┘ │
                        └───────────────────────────────────────┘
   guardrails and observability are either CROSS-CUTTING CONCERNS
   (teaching frame) or LAYERS OF THEIR OWN (production frame).
   Both framings are defensible. Say which one you are using.

SINGLE-AGENT                 MULTI-AGENT (orchestrator-worker)
   user                          user
    │                             │
    ▼                             ▼
 [ agent ]                  ┌────────────┐
  │  ▲   tools              │ ORCHESTRATOR│  (LLM decides who to call)
  ▼  │                      └──┬───┬───┬──┘
 tool/db                       │   │   │  (agent-as-tool calls)
                    ┌──────────┘   │   └──────────┐
                    ▼              ▼              ▼
               [flights]       [hotels]      [activities]  ...workers
               tools           tools          tools
                    └──────────┬──┴─────────────┘
                               ▼
                        synthesize → answer

4. How it works

The four layers

1. Reasoning layer. The LLM plus the prompting/reasoning strategy (ReAct, plan-and-execute, reflection — see the reasoning-patterns article). This is where "what should I do next?" is answered. Swappable: same architecture, different reasoning pattern.
2. Orchestration layer. The control machinery around the model: the agent loop, iteration cap, tool-call dispatch, error handling and retries, and the stop condition. In frameworks this is LangGraph's graph runner, Strands' Agent runtime, or your own while loop. It also owns state as it flows between steps.
3. Memory layer. Short-term = the running conversation/scratchpad in this task (message history, intermediate tool results). Long-term = persisted knowledge: past sessions, user profile, a vector store for retrieval. A conversation manager (e.g. summarizing old turns to fit the context window) lives here.
4. Tool-integration layer. The typed functions the agent can call — search APIs, databases, calculators, and other agents. This layer also enforces safety: schema validation, sandboxing, guardrails.

These layers exist in every agent, single or multi. Multi-agent systems just nest them: each worker is itself a full four-layer agent, and the orchestrator is a four-layer agent whose tools happen to be other agents.

Note on terminology: "four layers" is a teaching frame, not an official taxonomy. Anthropic's Building Effective Agents frames the building block as an "augmented LLM" — a model enhanced with retrieval, tools, and memory. Splitting out "reasoning" and "orchestration" is a convenient way to organize that idea for study, not a standard definition — so present it as "how I think about it," not "the official layers."

The two production layers

The four-layer frame describes what an agent is. It does not describe what makes an agent shippable. Walk through almost any production reference architecture — the hyperscaler agent platforms, the enterprise agent stacks shipped by hardware and data vendors — and two more components are promoted to first-class, alongside the four, not folded into them:

5. Guardrails. The policy and enforcement surface: input filters (injection, PII, off-topic), tool guards (argument validation, allow-lists, scoped credentials, approval for irreversible actions), and output checks (moderation, schema, grounding). The design decision that matters is the default: fail-open returns a best-effort answer when a check errors; fail-closed refuses. Regulated and irreversible actions need fail-closed, and that default has to be a property of the layer, not a line in a prompt — a prompt instruction is a request, a routed refusal is a guarantee.
6. Observability. Traces of every decision (which tool, which arguments, which worker, why it stopped), cost and token accounting per completed task, and durable run artifacts you can replay. Without it, a wrong final answer in a multi-agent system gives you no way to tell which of the loops went wrong.

Why promote them? Because they are the two that decide whether the system can run unattended. Note the circularity in the four-layer frame: layer 4 above says tool-integration "also enforces safety." That is true of a single agent with four tools. It stops being true the moment the orchestrator's tools are other agents, each with its own tool layer — safety then has to be enforced around the whole nest, not inside one node of it. The same argument applies to observability: per-agent logs do not compose into a trace of the system.

The honest position is that the field has not settled this. Treat the count as a framing choice you make deliberately: four layers plus two cross-cutting concerns is the cleaner teaching model; six layers is the model that matches how production platforms are actually organised and sold. Pick one, name it as a frame, and be able to defend why the other exists.

Single-agent vs multi-agent

Single-agent: one LLM with one tool belt handles planning, reasoning, and execution end to end. Simpler to build, deploy, debug, and reason about cost. It becomes a bottleneck when the tool belt grows huge (tool-selection accuracy drops as the number of tools climbs), when the prompt has to juggle conflicting concerns, or when subtasks could run in parallel.

Multi-agent: several agents, each with a narrow role and its own small tool set. Benefits: focused prompts (each specialist's system prompt is short and sharp), separation of concerns, parallelism, and independent testing/evolution of each specialist. Costs: coordination overhead, more LLM calls (higher latency and $), and hard failure modes — poor task decomposition and weak inter-agent communication are the classic pitfalls.

Orchestrator-worker (the pattern to know cold)

An orchestrator agent receives the user input and transforms it into a set of subtasks. Each subtask is routed to a worker agent that returns a local result. The orchestrator observes results and dynamically decides the next subtasks, looping until it can synthesize a final answer. It's designed for workflows where the problem structure emerges at runtime — the orchestrator can't know in advance whether a query needs one specialist or four.

The clean implementation is the agent-as-tool pattern (documented in the OpenAI Agents SDK — Agent.as_tool() / the "Manager" pattern — and in Strands): wrap each worker agent in a @tool function and hand those tools to the orchestrator. Now the orchestrator's tool-calling machinery is the delegation mechanism — the model "calls the budget agent" the same way it would call a calculator.

Key challenges the interviewer will push on: dynamic routing (which worker for which subtask), context continuity (remembering decisions across cycles), clean handoffs (passing enough context without leaking everything), and error handling (one worker fails — retry, reroute, or degrade?).

Other multi-agent shapes (know they exist): hierarchical (orchestrators of orchestrators), network/blackboard (agents read/write shared state), and critic/verifier (one agent generates, another checks). Orchestrator-worker is the default; the others are specializations.

5. Intuition (not standard formulas)

These are back-of-envelope mental models to reason about the tradeoff — not established or published formulas. In an interview, use the ideas, not the equations.

1. More tools = harder tool selection. Tool-selection accuracy tends to drop as the number of candidate tools grows — more options means more distractors per decision (a point Anthropic and others make when warning against overloading one agent with tools). Splitting one 20-tool agent into an orchestrator over four 5-tool workers means each decision faces fewer candidates (5 inside a worker, 4 at the orchestrator) — that is why decomposition can improve reliability. There is no official closed-form here; it's a qualitative trend, so say it as one.

2. Multi-agent multiplies LLM calls. This one is just call-counting arithmetic, not a special formula. For an orchestrator that makes R delegation rounds, calling on average w workers per round, each worker running s internal steps:

LLM calls  ≈  R · (1  +  w · s)
       ▲          ▲     ▲    ▲
       │          │     │    └ steps inside each worker's own loop
       │          │     └ workers invoked per round
       │          └ the orchestrator's own call each round
       └ number of orchestration rounds

The 1 + is the orchestrator tax you pay per round. Parallelism helps latency (run the w workers concurrently) but not token cost — you still pay for all w·s calls. That's the quantitative core of "multi-agent adds cost; make sure the accuracy gain is worth it."

6. Real code

Here is a trip-planning orchestrator — the orchestrator-worker / agent-as-tool pattern, on Strands + Bedrock (Claude 3.5 Sonnet).

# orchestrator.py  (example, abridged)
from strands import Agent, tool
from strands.models import BedrockModel
from strands.agent.conversation_manager import SummarizingConversationManager

from agents.budget_agent import budget_agent
from agents.flights_agent import flights_discovery_agent
from agents.hotels_agent import hotels_agent
from agents.activities_agent import activities_agent

bedrock_model = BedrockModel(
    model_id="anthropic.claude-3-5-sonnet-20241022-v2:0",
    region_name=os.getenv("AWS_REGION", "us-east-1"),
    guardrail_id=get_guardrail_id(),          # tool-integration layer: a safety guardrail
)

# --- agent-as-tool: each SPECIALIST agent is wrapped as a TOOL ---
@tool
def budget_agent_tool(query: str) -> str:
    """Handle budget and price-related queries."""
    try:
        return str(budget_agent(query))       # a full four-layer agent, invoked as a tool
    except Exception as e:
        return f"Budget agent error: {str(e)}"  # error handling at the boundary

@tool
def hotels_agent_tool(query: str) -> str:
    """Handle hotel and lodging preference queries."""
    try:
        return str(hotels_agent(query))
    except Exception as e:
        return f"Hotels agent error: {str(e)}"

# ... flights_discovery_agent_tool, activities_agent_tool likewise ...

# memory layer: summarize old turns so long chats fit the context window
conversation_manager = SummarizingConversationManager(
    summary_ratio=0.3, preserve_recent_messages=5,
)

# reasoning + orchestration layer: ONE orchestrator whose tools ARE the specialists
orchestrator_agent = Agent(
    model=bedrock_model,
    system_prompt=ORCHESTRATOR_PROMPT,   # "coordinate these 4 specialists, synthesize"
    tools=[budget_agent_tool, flights_discovery_agent_tool,
           hotels_agent_tool, activities_agent_tool],
    conversation_manager=conversation_manager,
)

def process_query(query: str) -> str:
    return str(orchestrator_agent(query))    # the orchestrator DECIDES which specialists to call

And a worker is just a smaller four-layer agent with its own tool and a focused prompt:

# budget_agent.py  (example)
from strands import Agent
from strands.models import BedrockModel
from agents.tools.budget_tools import calculate_budget

budget_agent = Agent(
    model=bedrock_model,
    system_prompt=BUDGET_AGENT_PROMPT,       # short, sharp: "you are a budget analysis agent"
    tools=[calculate_budget],                # ONE narrow tool, not twenty
)

Notice the layers map 1:1: BedrockModel = reasoning, Agent(...) runtime = orchestration, SummarizingConversationManager = memory, tools=[...] + guardrail = tool-integration.

7. Real-world example

A trip-planning assistant is a textbook orchestrator-worker system. A query like "a 5-day trip for two under $2000 with good weather, and we want to avoid long layovers" flows like this:

1. The orchestrator (Claude 3.5 Sonnet) parses the request and extracts the layover constraint.
2. It decides at runtime which specialists to delegate to — here likely all four: a budget agent (price filter), a flights agent (search/discovery), a hotels agent (lodging match + preference filtering), and an activities agent (things to do). A simpler query ("cheapest weekend getaway nearby") might invoke only two.
3. Each specialist runs its own mini-loop with its own tools (calculate_budget, search_flights, etc.).
4. The orchestrator synthesizes the four local results into one coherent, constraint-satisfying recommendation, guarded by a Bedrock guardrail.

Contrast three shapes of the same trip-planning domain — a good way to show you know when to add agents:

• Single-agent workflow (one LCEL/LangGraph chain, no delegation) — you don't need multi-agent for trip recs; it's the simplest and cheapest choice when the concerns aren't really separable.
• Orchestrator-worker (above) — chosen when the concerns (budget math, flight search, hotel matching, activity planning) are genuinely separable and benefit from focused prompts and parallel specialists.
• Retrieval-specialist variant — a supervisor decomposes a complex query and routes it to k specialist retriever agents running in parallel over a vector store; orchestrator-worker where the workers are RAG specialists.
• Fixed multi-step pipeline — when the steps are known in advance, a fixed multi-node graph beats a free-roaming multi-agent team on reliability (it's not "agentic," but it's often the right, most reliable call).

8. Interview questions companies actually ask

Q1. What are the core layers of an agent architecture? [easy] Reasoning (LLM decides), orchestration (loop, routing, retries, stop), memory (short + long term), and tool-integration (APIs, DBs, other agents, plus safety). Every agent has all four; multi-agent systems nest them. (Note: this is a teaching frame; Anthropic's official framing is the "augmented LLM" — model + retrieval + tools + memory.) (GeeksforGeeks)

Q2. Single-agent vs multi-agent — when do you pick each? [medium] Single-agent for simpler, sequential tasks: easier to build, cheaper, debuggable. Multi-agent when concerns are separable, the tool belt is too large for one prompt, or subtasks can run in parallel — at the cost of coordination overhead and more LLM calls. Don't add a second agent unless it buys accuracy or parallelism worth the latency/$. (ProjectPro)

Q3. Explain the orchestrator-worker pattern. [medium] An orchestrator turns the input into subtasks, routes each to a specialized worker, observes the local results, and dynamically decides the next subtasks until it can synthesize a final answer. It fits problems whose structure only emerges at runtime. (Anthropic)

Q4. What is the "agent-as-tool" pattern? [medium] Wrap each worker agent in a tool function and expose those tools to the orchestrator. The orchestrator's normal tool-calling mechanism becomes the delegation mechanism — calling a sub-agent looks identical to calling a calculator. For example, a @tool budget_agent_tool(...) wrapper does exactly this. (Documented in the OpenAI Agents SDK and Strands.)

Q5. What are the main failure modes of multi-agent systems? [hard] Poor task decomposition (subtasks overlap or leave gaps) and inadequate inter-agent communication (context lost across handoffs). Add: coordination deadlocks, cascading errors when one worker fails, and cost/latency blowup from too many rounds. Mitigate with clear role boundaries, structured handoff schemas, per-worker error handling, and round caps. (ProjectPro)

Q6. Why can splitting one big agent into specialists improve accuracy? [hard] Tool-selection accuracy degrades as the number of candidate tools grows (more distractors per decision). Decomposition shrinks the tool set visible at each decision point — 5 tools inside a worker, 4 at the orchestrator — so each decision faces fewer distractors, plus each specialist gets a shorter, sharper prompt. (This is a qualitative trend, not a formula.)

Q7. Name orchestration frameworks and what they give you. [easy] LangGraph (graph/state-machine orchestration), CrewAI (role-based crews), AutoGen (conversational multi-agent), Strands (agent runtime, agent-as-tool), plus infra like Ray/Airflow. They provide the orchestration layer: looping, routing, state, retries, and (some) memory management. (ProjectPro)

Q8. Does multi-agent parallelism reduce cost? [hard] It reduces latency (run workers concurrently) but not token cost — you still pay for every worker's calls, plus the per-round orchestrator tax. Roughly calls ≈ R·(1 + w·s) (just call-counting, not a special formula). If accuracy doesn't improve, multi-agent is pure overhead.

Q9. How does memory differ across the architecture? [medium] Short-term memory is the in-task scratchpad (message history, tool results) and lives in the orchestration state; long-term memory is persisted (user profile, past sessions, vector store). A conversation manager (e.g. a SummarizingConversationManager) compresses old turns so long tasks fit the context window.

Q10. Critic/verifier vs orchestrator-worker — when? [hard] Orchestrator-worker distributes different subtasks to specialists. Critic/verifier has one agent generate and another check the same output — use it when correctness matters more than decomposition (code that must compile, claims that must be grounded). They compose: an orchestrator can route to a worker whose job is verification.

Q11. Where do guardrails and observability live in your architecture? [medium] Either as cross-cutting concerns over the four layers, or as two first-class layers of their own — and the answer an interviewer is listening for is that you know it is a framing choice rather than a fact. The reason to promote them is composition: a single agent can enforce safety inside its tool-integration layer, but once the orchestrator's tools are other agents, each with its own tool layer, safety and tracing have to wrap the whole nest, and per-agent logs do not compose into a system trace. The concrete artifacts to name are input guard, tool guard with scoped credentials and approval for irreversible actions, a fail-closed default, and on the other side traces, cost per completed task, and replayable run artifacts.

9. When to use / tradeoffs

Rules of thumb: start single-agent; split into orchestrator-worker only when a specialist's concern is genuinely independent (separate tools, separate prompt) or you need parallelism. Keep each worker's tool set small. Define explicit handoff schemas and per-worker error handling. Always cap orchestration rounds and inner steps.

Honest limits. The layer counts in this article are teaching models, not a specification — no standards body defines them, and published vendor architectures disagree on whether guardrails and observability are layers or concerns. The claim that decomposition improves tool selection is a qualitative trend observed in practice, not a measured law, and it reverses when the orchestrator itself faces many workers. The call-count arithmetic assumes every worker is invoked once per round; retries and reflection loops break that assumption and are the usual reason a multi-agent system costs far more than its design estimate.

10. Summary + related articles

An agent is four layers — reasoning, orchestration, memory, tool-integration — and you scale it either as a single agent or, when concerns separate cleanly, as an orchestrator-worker multi-agent team using the agent-as-tool pattern. Multi-agent buys focus, parallelism, and reliability (fewer distractors per decision) at the price of coordination overhead and more LLM calls. Decompose only when the accuracy or parallelism gain outweighs that cost. Whichever shape you choose, decide explicitly whether guardrails and observability are cross-cutting concerns or layers in their own right — nesting agents inside agents is what forces that question, because safety and tracing then have to wrap the nest rather than sit inside one node.

Related articles:

• What Are Agents — agent vs workflow vs plain LLM; who decides the next step.
• Reasoning Patterns — the reasoning layer in depth (ReAct, Reflexion, Plan-and-Execute, ReWOO, ToT).
• Tool Use — the tool-integration layer: schemas, structured output, reliability.

Sources: Anthropic — Building Effective Agents (orchestrator-workers, augmented LLM) · OpenAI Agents SDK — Agents as Tools / orchestration · ProjectPro — Agentic AI Interview Q&A · GeeksforGeeks — Agentic AI Interview Questions