Agent Architectures

1. TL;DR

An agent architecture is the arrangement of four layers — reasoning, orchestration, memory, and tool-integration — plus the choice of how many agents you use and how they coordinate (single agent vs multi-agent; and among multi-agent, the dominant pattern is orchestrator-worker). It matters because the architecture, not the model, is what decides whether your system is debuggable, cheap, and correct at scale — interviewers probe this to see if you can tell when a second agent helps versus when it just adds latency and coordination bugs.

2. Simple explanation

Think of an agent as a person doing a job:

• Reasoning layer = their brain — deciding what to do next.
• Orchestration layer = their calendar and to-do list — sequencing steps, handling retries, deciding when to stop.
• Memory layer = their notebook — what they've learned this session and across sessions.
• Tool-integration layer = their hands and phone — the APIs, databases, and functions they can actually operate.

A single-agent system is one skilled generalist doing the whole job alone. A multi-agent system is a team: a manager (orchestrator) who understands the request and delegates to specialists (workers) — a flights expert, a hotels expert, an activities planner — then stitches their answers into one recommendation. The manager doesn't book or pack; it coordinates. That "manager + specialists" shape is the orchestrator-worker pattern, and it's the multi-agent architecture you'll be asked about most.

3. Diagram

THE FOUR LAYERS (inside ONE agent)
┌───────────────────────────────────────────────┐
│  REASONING      LLM decides next action        │
│      ▲  │                                       │
│      │  ▼                                       │
│  ORCHESTRATION  loop · routing · retries · stop │
│      ▲  │                    ▲                  │
│      │  ▼                    │                  │
│  TOOL-INTEGRATION      MEMORY                   │
│  search · db · APIs    short-term + long-term   │
└───────────────────────────────────────────────┘

SINGLE-AGENT                 MULTI-AGENT (orchestrator-worker)
   user                          user
    │                             │
    ▼                             ▼
 [ agent ]                  ┌────────────┐
  │  ▲   tools              │ ORCHESTRATOR│  (LLM decides who to call)
  ▼  │                      └──┬───┬───┬──┘
 tool/db                       │   │   │  (agent-as-tool calls)
                    ┌──────────┘   │   └──────────┐
                    ▼              ▼              ▼
               [flights]       [hotels]      [activities]  ...workers
               tools           tools          tools
                    └──────────┬──┴─────────────┘
                               ▼
                        synthesize → answer

4. How it works

The four layers

1. Reasoning layer. The LLM plus the prompting/reasoning strategy (ReAct, plan-and-execute, reflection — see the reasoning-patterns article). This is where "what should I do next?" is answered. Swappable: same architecture, different reasoning pattern.
2. Orchestration layer. The control machinery around the model: the agent loop, iteration cap, tool-call dispatch, error handling and retries, and the stop condition. In frameworks this is LangGraph's graph runner, Strands' Agent runtime, or your own while loop. It also owns state as it flows between steps.
3. Memory layer. Short-term = the running conversation/scratchpad in this task (message history, intermediate tool results). Long-term = persisted knowledge: past sessions, user profile, a vector store for retrieval. A conversation manager (e.g. summarizing old turns to fit the context window) lives here.
4. Tool-integration layer. The typed functions the agent can call — search APIs, databases, calculators, and other agents. This layer also enforces safety: schema validation, sandboxing, guardrails.

These layers exist in every agent, single or multi. Multi-agent systems just nest them: each worker is itself a full four-layer agent, and the orchestrator is a four-layer agent whose tools happen to be other agents.

Note on terminology: "four layers" is a teaching frame, not an official taxonomy. Anthropic's Building Effective Agents frames the building block as an "augmented LLM" — a model enhanced with retrieval, tools, and memory. Splitting out "reasoning" and "orchestration" is a convenient way to organize that idea for study, not a standard definition — so present it as "how I think about it," not "the official layers."

Single-agent vs multi-agent

Single-agent: one LLM with one tool belt handles planning, reasoning, and execution end to end. Simpler to build, deploy, debug, and reason about cost. It becomes a bottleneck when the tool belt grows huge (tool-selection accuracy drops as the number of tools climbs), when the prompt has to juggle conflicting concerns, or when subtasks could run in parallel.

Multi-agent: several agents, each with a narrow role and its own small tool set. Benefits: focused prompts (each specialist's system prompt is short and sharp), separation of concerns, parallelism, and independent testing/evolution of each specialist. Costs: coordination overhead, more LLM calls (higher latency and $), and hard failure modes — poor task decomposition and weak inter-agent communication are the classic pitfalls.

Orchestrator-worker (the pattern to know cold)

An orchestrator agent receives the user input and transforms it into a set of subtasks. Each subtask is routed to a worker agent that returns a local result. The orchestrator observes results and dynamically decides the next subtasks, looping until it can synthesize a final answer. It's designed for workflows where the problem structure emerges at runtime — the orchestrator can't know in advance whether a query needs one specialist or four.

The clean implementation is the agent-as-tool pattern (documented in the OpenAI Agents SDK — Agent.as_tool() / the "Manager" pattern — and in Strands): wrap each worker agent in a @tool function and hand those tools to the orchestrator. Now the orchestrator's tool-calling machinery is the delegation mechanism — the model "calls the budget agent" the same way it would call a calculator.

Key challenges the interviewer will push on: dynamic routing (which worker for which subtask), context continuity (remembering decisions across cycles), clean handoffs (passing enough context without leaking everything), and error handling (one worker fails — retry, reroute, or degrade?).

Other multi-agent shapes (know they exist): hierarchical (orchestrators of orchestrators), network/blackboard (agents read/write shared state), and critic/verifier (one agent generates, another checks). Orchestrator-worker is the default; the others are specializations.

5. Intuition (not standard formulas)

These are back-of-envelope mental models to reason about the tradeoff — not established or published formulas. In an interview, use the ideas, not the equations.

1. More tools = harder tool selection. Tool-selection accuracy tends to drop as the number of candidate tools grows — more options means more distractors per decision (a point Anthropic and others make when warning against overloading one agent with tools). Splitting one 20-tool agent into an orchestrator over four 5-tool workers means each decision faces fewer candidates (5 inside a worker, 4 at the orchestrator) — that is why decomposition can improve reliability. There is no official closed-form here; it's a qualitative trend, so say it as one.

2. Multi-agent multiplies LLM calls. This one is just call-counting arithmetic, not a special formula. For an orchestrator that makes R delegation rounds, calling on average w workers per round, each worker running s internal steps:

LLM calls  ≈  R · (1  +  w · s)
       ▲          ▲     ▲    ▲
       │          │     │    └ steps inside each worker's own loop
       │          │     └ workers invoked per round
       │          └ the orchestrator's own call each round
       └ number of orchestration rounds

The 1 + is the orchestrator tax you pay per round. Parallelism helps latency (run the w workers concurrently) but not token cost — you still pay for all w·s calls. That's the quantitative core of "multi-agent adds cost; make sure the accuracy gain is worth it."

6. Real code

Here is a trip-planning orchestrator — the orchestrator-worker / agent-as-tool pattern, on Strands + Bedrock (Claude 3.5 Sonnet).

# orchestrator.py  (example, abridged)
from strands import Agent, tool
from strands.models import BedrockModel
from strands.agent.conversation_manager import SummarizingConversationManager

from agents.budget_agent import budget_agent
from agents.flights_agent import flights_discovery_agent
from agents.hotels_agent import hotels_agent
from agents.activities_agent import activities_agent

bedrock_model = BedrockModel(
    model_id="anthropic.claude-3-5-sonnet-20241022-v2:0",
    region_name=os.getenv("AWS_REGION", "us-east-1"),
    guardrail_id=get_guardrail_id(),          # tool-integration layer: a safety guardrail
)

# --- agent-as-tool: each SPECIALIST agent is wrapped as a TOOL ---
@tool
def budget_agent_tool(query: str) -> str:
    """Handle budget and price-related queries."""
    try:
        return str(budget_agent(query))       # a full four-layer agent, invoked as a tool
    except Exception as e:
        return f"Budget agent error: {str(e)}"  # error handling at the boundary

@tool
def hotels_agent_tool(query: str) -> str:
    """Handle hotel and lodging preference queries."""
    try:
        return str(hotels_agent(query))
    except Exception as e:
        return f"Hotels agent error: {str(e)}"

# ... flights_discovery_agent_tool, activities_agent_tool likewise ...

# memory layer: summarize old turns so long chats fit the context window
conversation_manager = SummarizingConversationManager(
    summary_ratio=0.3, preserve_recent_messages=5,
)

# reasoning + orchestration layer: ONE orchestrator whose tools ARE the specialists
orchestrator_agent = Agent(
    model=bedrock_model,
    system_prompt=ORCHESTRATOR_PROMPT,   # "coordinate these 4 specialists, synthesize"
    tools=[budget_agent_tool, flights_discovery_agent_tool,
           hotels_agent_tool, activities_agent_tool],
    conversation_manager=conversation_manager,
)

def process_query(query: str) -> str:
    return str(orchestrator_agent(query))    # the orchestrator DECIDES which specialists to call

And a worker is just a smaller four-layer agent with its own tool and a focused prompt:

# budget_agent.py  (example)
from strands import Agent
from strands.models import BedrockModel
from agents.tools.budget_tools import calculate_budget

budget_agent = Agent(
    model=bedrock_model,
    system_prompt=BUDGET_AGENT_PROMPT,       # short, sharp: "you are a budget analysis agent"
    tools=[calculate_budget],                # ONE narrow tool, not twenty
)

Notice the layers map 1:1: BedrockModel = reasoning, Agent(...) runtime = orchestration, SummarizingConversationManager = memory, tools=[...] + guardrail = tool-integration.

7. Real-world example

A trip-planning assistant is a textbook orchestrator-worker system. A query like "a 5-day trip for two under $2000 with good weather, and we want to avoid long layovers" flows like this:

1. The orchestrator (Claude 3.5 Sonnet) parses the request and extracts the layover constraint.
2. It decides at runtime which specialists to delegate to — here likely all four: a budget agent (price filter), a flights agent (search/discovery), a hotels agent (lodging match + preference filtering), and an activities agent (things to do). A simpler query ("cheapest weekend getaway nearby") might invoke only two.
3. Each specialist runs its own mini-loop with its own tools (calculate_budget, search_flights, etc.).
4. The orchestrator synthesizes the four local results into one coherent, constraint-satisfying recommendation, guarded by a Bedrock guardrail.

Contrast three shapes of the same trip-planning domain — a good way to show you know when to add agents:

• Single-agent workflow (one LCEL/LangGraph chain, no delegation) — you don't need multi-agent for trip recs; it's the simplest and cheapest choice when the concerns aren't really separable.
• Orchestrator-worker (above) — chosen when the concerns (budget math, flight search, hotel matching, activity planning) are genuinely separable and benefit from focused prompts and parallel specialists.
• Retrieval-specialist variant — a supervisor decomposes a complex query and routes it to k specialist retriever agents running in parallel over a vector store; orchestrator-worker where the workers are RAG specialists.
• Fixed multi-step pipeline — when the steps are known in advance, a fixed multi-node graph beats a free-roaming multi-agent team on reliability (it's not "agentic," but it's often the right, most reliable call).

8. Interview questions companies actually ask

Q1. What are the core layers of an agent architecture? [easy] Reasoning (LLM decides), orchestration (loop, routing, retries, stop), memory (short + long term), and tool-integration (APIs, DBs, other agents, plus safety). Every agent has all four; multi-agent systems nest them. (Note: this is a teaching frame; Anthropic's official framing is the "augmented LLM" — model + retrieval + tools + memory.) (GeeksforGeeks)

Q2. Single-agent vs multi-agent — when do you pick each? [medium] Single-agent for simpler, sequential tasks: easier to build, cheaper, debuggable. Multi-agent when concerns are separable, the tool belt is too large for one prompt, or subtasks can run in parallel — at the cost of coordination overhead and more LLM calls. Don't add a second agent unless it buys accuracy or parallelism worth the latency/$. (ProjectPro)

Q3. Explain the orchestrator-worker pattern. [medium] An orchestrator turns the input into subtasks, routes each to a specialized worker, observes the local results, and dynamically decides the next subtasks until it can synthesize a final answer. It fits problems whose structure only emerges at runtime. (Anthropic)

Q4. What is the "agent-as-tool" pattern? [medium] Wrap each worker agent in a tool function and expose those tools to the orchestrator. The orchestrator's normal tool-calling mechanism becomes the delegation mechanism — calling a sub-agent looks identical to calling a calculator. For example, a @tool budget_agent_tool(...) wrapper does exactly this. (Documented in the OpenAI Agents SDK and Strands.)

Q5. What are the main failure modes of multi-agent systems? [hard] Poor task decomposition (subtasks overlap or leave gaps) and inadequate inter-agent communication (context lost across handoffs). Add: coordination deadlocks, cascading errors when one worker fails, and cost/latency blowup from too many rounds. Mitigate with clear role boundaries, structured handoff schemas, per-worker error handling, and round caps. (ProjectPro)

Q6. Why can splitting one big agent into specialists improve accuracy? [hard] Tool-selection accuracy degrades as the number of candidate tools grows (more distractors per decision). Decomposition shrinks the tool set visible at each decision point — 5 tools inside a worker, 4 at the orchestrator — so each decision faces fewer distractors, plus each specialist gets a shorter, sharper prompt. (This is a qualitative trend, not a formula.)

Q7. Name orchestration frameworks and what they give you. [easy] LangGraph (graph/state-machine orchestration), CrewAI (role-based crews), AutoGen (conversational multi-agent), Strands (agent runtime, agent-as-tool), plus infra like Ray/Airflow. They provide the orchestration layer: looping, routing, state, retries, and (some) memory management. (ProjectPro)

Q8. Does multi-agent parallelism reduce cost? [hard] It reduces latency (run workers concurrently) but not token cost — you still pay for every worker's calls, plus the per-round orchestrator tax. Roughly calls ≈ R·(1 + w·s) (just call-counting, not a special formula). If accuracy doesn't improve, multi-agent is pure overhead.

Q9. How does memory differ across the architecture? [medium] Short-term memory is the in-task scratchpad (message history, tool results) and lives in the orchestration state; long-term memory is persisted (user profile, past sessions, vector store). A conversation manager (e.g. a SummarizingConversationManager) compresses old turns so long tasks fit the context window.

Q10. Critic/verifier vs orchestrator-worker — when? [hard] Orchestrator-worker distributes different subtasks to specialists. Critic/verifier has one agent generate and another check the same output — use it when correctness matters more than decomposition (code that must compile, claims that must be grounded). They compose: an orchestrator can route to a worker whose job is verification.

9. When to use / tradeoffs

Rules of thumb: start single-agent; split into orchestrator-worker only when a specialist's concern is genuinely independent (separate tools, separate prompt) or you need parallelism. Keep each worker's tool set small. Define explicit handoff schemas and per-worker error handling. Always cap orchestration rounds and inner steps.

10. Summary + related articles

An agent is four layers — reasoning, orchestration, memory, tool-integration — and you scale it either as a single agent or, when concerns separate cleanly, as an orchestrator-worker multi-agent team using the agent-as-tool pattern. Multi-agent buys focus, parallelism, and reliability (fewer distractors per decision) at the price of coordination overhead and more LLM calls. Decompose only when the accuracy or parallelism gain outweighs that cost.

Related articles:

• What Are Agents — agent vs workflow vs plain LLM; who decides the next step.
• Reasoning Patterns — the reasoning layer in depth (ReAct, Reflexion, Plan-and-Execute, ReWOO, ToT).
• Tool Use — the tool-integration layer: schemas, structured output, reliability.

Sources: Anthropic — Building Effective Agents (orchestrator-workers, augmented LLM) · OpenAI Agents SDK — Agents as Tools / orchestration · ProjectPro — Agentic AI Interview Q&A · GeeksforGeeks — Agentic AI Interview Questions