Multi-Agent Patterns

TL;DR

• A multi-agent system (MAS) is several LLM-driven agents, each with its own prompt/tools/context window, coordinated to solve one task.
• The patterns worth memorizing: orchestrator–worker (a.k.a. supervisor / agent-as-tool), hierarchical, sequential (pipeline), parallel (fan-out/fan-in), network (swarm / peer-to-peer), debate, and evaluator–optimizer.
• Pick a pattern by the shape of the work: distinct specialties → orchestrator-worker; ordered dependencies → sequential; independent sub-questions → parallel; quality-critical generation → evaluator-optimizer.
• Senior insight (Anthropic): most tasks that look multi-agent are better as a single ReAct agent with good tools. Multi-agent buys you parallelism, isolation, and specialization — but you pay in tokens (~15× a chat), latency, coordination code, and new failure modes. Use it only when the work is genuinely separable.

---

Simple explanation + analogy

Think of a professional kitchen.

• A single agent is one line cook who does everything: takes the order, chops, grills, plates. Fine for a small operation.
• An orchestrator–worker kitchen has a head chef (expediter) who reads the ticket and calls out to stations — grill, sauté, pastry. Each station is a specialist with its own tools and knowledge. The expediter never grills; they route and plate the final dish.
• Sequential is an assembly line: prep → cook → plate, each step depends on the last.
• Parallel is three stations cooking three dishes of the same order at once, then plating together.
• Debate is two chefs arguing whether the sauce needs more acid, with the head chef deciding.
• Evaluator–optimizer is a cook plating, a critic tasting and sending it back, the cook re-plating until it passes.

The head chef adds coordination overhead. In a two-item order you don't need one — you'd just slow things down. That is the entire multi-agent tradeoff in one image.

---

Diagram

ORCHESTRATOR–WORKER (supervisor / agent-as-tool)      SEQUENTIAL (pipeline)
                                                       
          ┌─────────────┐                              in ─▶ [A] ─▶ [B] ─▶ [C] ─▶ out
   user ─▶│ ORCHESTRATOR│◀── synthesizes final                (each depends on prior)
          └──────┬──────┘
      routes │   │   │   │   (agent-as-tool calls)     PARALLEL (fan-out / fan-in)
         ┌───┘   │   │   └────┐                                 ┌▶ [A] ┐
         ▼       ▼   ▼        ▼                          in ─▶ split ─▶ [B] ─▶ join ─▶ out
      flights hotels activities budget                          └▶ [C] ┘
      agent   agent  agent      agent                    (independent sub-tasks, then merge)

HIERARCHICAL (teams of teams)          NETWORK / SWARM         DEBATE + JUDGE
   ┌────────┐                          [A]◀──▶[B]              [pro]─┐
   │ top    │                            ▲     │               ...   ├▶[judge]▶ answer
   └─┬────┬─┘                            │     ▼              [con]─┘
     ▼    ▼                            [C]◀──▶[D]             EVALUATOR–OPTIMIZER
  ┌lead┐ ┌lead┐    (any agent may hand off             [generate]⇄[evaluate] loop
  ▼  ▼   ▼  ▼       to any peer directly)               until "pass" or max rounds
 wkr wkr wkr wkr

---

How it works (deep)

1. Orchestrator–worker (supervisor / agent-as-tool)

A central orchestrator LLM receives the request, decides which specialist(s) to invoke, calls them, and synthesizes their outputs into a single answer. The workers are domain experts, each with a narrow system prompt and its own toolset.

Two common implementations:

• Agent-as-tool — each worker is wrapped as a tool the orchestrator can call. The orchestrator is just a ReAct agent whose "tools" happen to be other agents. This is what the concierge does (see the code section). Clean, composable, and the framework handles the loop.
• Router/handoff — the orchestrator emits a routing decision ("go to hotels_agent") and control transfers; the worker's output either returns to the orchestrator or ends the run.

Strength: clean separation, easy to add a new specialist. Weakness: the orchestrator is a single point of failure — one bad routing decision degrades the whole pipeline, and every hop costs a full LLM round-trip.

2. Hierarchical (teams of teams)

Orchestrator-worker taken to N levels: a top orchestrator delegates to team leads, each of which is itself an orchestrator over its own workers. Use when the task tree is genuinely deep (e.g., "plan an entire product launch" → marketing lead, eng lead, finance lead, each with sub-agents). Rarely needed; the coordination cost compounds per level.

3. Sequential (pipeline)

Agents/nodes run in a fixed order; each consumes the previous one's output. This is the backbone of a fixed multi-step grading pipeline: ingest_ocr → load_rubric → query_deduction_mem → grade_attempt → self_consistency → …. It's not really "many autonomous agents" — it's an orchestrated workflow where the LLM does judgment at each node and Python controls flow. That determinism is a feature.

4. Parallel (fan-out / fan-in)

Split independent sub-tasks, run them concurrently, then merge. Two variants:

• Same task, many items (data parallelism): a grading pipeline grades N students concurrently under an asyncio.Semaphore.
• Different sub-questions, one query (task parallelism): the clinical RAG system routes one clinical question to a subset of its subspecialty agents in parallel, then synthesizes a consensus.

This is the pattern Anthropic's research system leans on — breadth-first questions decompose into independent strands, and parallelism explores a larger search space than one agent can.

5. Network / swarm (peer-to-peer)

No central boss. Any agent can hand off to any peer via a Command, and whoever holds the turn responds or delegates. More flexible, but routing is emergent and much harder to reason about, test, and bound. Good for open-ended collaboration; risky in production where you need guarantees.

6. Debate

Multiple agents argue different positions on the same question; a judge agent picks the strongest answer (or synthesizes). Adversarial critique surfaces weaknesses and reduces hallucination on hard reasoning problems. Cost scales with rounds × debaters, so reserve it for high-stakes reasoning, not routine tasks.

7. Evaluator–optimizer

One agent generates, a second evaluates against explicit criteria and returns feedback, and the generator revises — looping until "pass" or a round cap. This is a quality pattern (writing, code, structured extraction). The clinical RAG system's schema-validated consensus extraction and a grading pipeline's self-consistency confidence gate are production cousins: generate → check → flag/retry.

Choosing — the decision table

---

The math

Multi-agent design is largely a cost/latency/reliability calculation. Two back-of-envelope models interviewers like.

Token & latency cost. For an orchestrator that calls k workers, each worker doing rᵢ reasoning turns:

total_tokens ≈ T_orch + Σ_{i=1..k} (T_handoff_in + rᵢ · T_worker_turn + T_handoff_out)

Every handoff re-serializes context, so tokens grow super-linearly with agents. Anthropic reports multi-agent runs use ~15× the tokens of a plain chat — that multiplier is the interview answer to "why not always multi-agent?"

Latency. Sequential of k steps: L ≈ Σ Lᵢ (adds up). Parallel of k independent steps: L ≈ max(Lᵢ) + L_merge (bounded by the slowest branch). That gap is the whole reason to parallelize.

Reliability compounding. If each agent/step succeeds independently with probability p, a chain of n steps succeeds with only:

P(success) = pⁿ

At p = 0.95, n = 10 → 0.60. This is why long agent chains need checkpoints, retries, and confidence gates (self-consistency's cv < cv_threshold) rather than one long unguarded run.

---

Code

Orchestrator with agents-as-tools — the concierge

The concierge wraps four specialists as @tool functions and hands them to a Strands orchestrator running on Bedrock Claude. The orchestrator never does budget math or flight searches itself — it routes and synthesizes.

# orchestrator.py  (example — Strands + Bedrock, Claude 3.5 Sonnet)
from strands import Agent, tool
from strands.models import BedrockModel
from strands.agent.conversation_manager import SummarizingConversationManager

from agents.flights_agent import flights_agent
from agents.hotels_agent import hotels_agent
from agents.activities_agent import activities_agent
from agents.budget_agent import budget_agent
from agents.utils.guardrail import get_guardrail_id

bedrock_model = BedrockModel(
    model_id="anthropic.claude-3-5-sonnet-20241022-v2:0",
    region_name=os.getenv("AWS_REGION", "us-east-1"),
    guardrail_id=get_guardrail_id(),           # Bedrock guardrail on every call
)

@tool
def activities_agent_tool(query: str) -> str:
    """Handle sightseeing and activity matching queries."""
    try:
        return str(activities_agent(query))   # agent-AS-tool: a whole agent behind a tool
    except Exception as e:
        return f"Activities agent error: {str(e)}"     # errors become data, not crashes

# ... flights_agent_tool, hotels_agent_tool, budget_agent_tool defined the same way

orchestrator_agent = Agent(
    model=bedrock_model,
    system_prompt=ORCHESTRATOR_PROMPT,             # "you coordinate 4 specialists ..."
    tools=[flights_agent_tool, hotels_agent_tool,
           activities_agent_tool, budget_agent_tool],
    conversation_manager=SummarizingConversationManager(
        summary_ratio=0.3, preserve_recent_messages=5),  # compress history to fight context bloat
)

def process_query(query: str) -> str:
    return str(orchestrator_agent(query))

The specialist behind that tool is itself a minimal ReAct agent — narrow prompt, narrow tools:

# activities_agent.py  (example)
activities_agent = Agent(
    model=bedrock_model,
    system_prompt=ACTIVITIES_AGENT_PROMPT,   # "rank activities; filter by weather FIRST"
    tools=[generate_activity_ranking_tool, filter_activities_by_weather_hybrid],
)

Two design choices worth calling out in an interview: (1) each worker owns its own system prompt and tool list — that isolation is the point of going multi-agent; (2) the tool wrappers catch exceptions and return strings, so a worker failure becomes an observation the orchestrator can react to, not a pipeline crash.

Parallel subset-of-N routing — the clinical RAG system

The supervisor exposes one tool per subspecialty and lets the model pick which to consult; retrieval runs synchronously inside each tool so the model can't skip the evidence search.

# parallel_specialist_tools.py  (example)
def _specialist_tool(specialty: str, description: str):
    @tool(name=_specialist_tool_name(specialty), description=description)
    async def specialist_impl(query: str):
        async for event in _consult_specialist(specialty, query):
            yield event                          # stream papers first, then synthesis
    return specialist_impl

burn_trauma_specialist = _specialist_tool("burn_trauma", "Consult the Burn Surgery specialist ...")
hand_surgery_specialist = _specialist_tool("hand_surgery", "Consult the Hand Surgery specialist ...")
# ... several specialists total; the supervisor routes a few of them per query

Sequential pipeline — a grading pipeline

# grading_graph.py  (example — LangGraph)
g.add_edge("ingest_ocr", "load_rubric")
g.add_edge("load_rubric", "query_deduction_mem")
g.add_edge("query_deduction_mem", "grade_attempt")
g.add_edge("grade_attempt", "self_consistency")
g.add_conditional_edges("self_consistency", route_after_confidence)  # branch on confidence

---

Real-world example

The concierge (orchestrator–worker). "Plan a 5-day trip under $2000 with good weather." The orchestrator extracts the budget cap, calls flights_agent_tool (search_flights), hotels_agent_tool (find_hotels), activities_agent_tool (weather filter before ranking — good-weather first), and budget_agent_tool (calculate_budget), then synthesizes one itinerary. Four distinct specialties, one answer — a textbook fit for the pattern.

The clinical RAG system (parallel subset-of-N). "Grafting technique for pediatric burns?" → supervisor consults burn_trauma and general specialists in parallel over a large Pinecone index, then a consensus-extraction pass returns structured agreement/debate/emerging findings with citations. Parallelism here is justified: subspecialties are independent evidence strands.

A grading pipeline (sequential + parallel batch). Single-student grading is a strict pipeline; batch grading fans out across students under a semaphore with a calibration phase and a circuit breaker that aborts if the early error rate is too high. "LLM points, Python reads" — the model judges each answer, Python owns flow, scoring, and gates.

The team-chat suggester (sequential graph with routing). A team-chat suggester: classify_intent → classify_request → {retrieve→suggest | answer_followup | create_task}. It deliberately stays a small routed graph, not a swarm — easy to test.

---

Interview questions companies actually ask

1. What is the orchestrator–worker (supervisor) pattern, and what's its main weakness? [easy] A central LLM decomposes the request, delegates to specialist workers, and synthesizes their outputs. Main weakness: the orchestrator is a single point of failure — a bad routing decision degrades everything, and every hop is a full LLM round-trip (cost + latency). See TrueFoundry: Multi-Agent Architecture.

2. When would you NOT use a multi-agent system? [medium] When the task is one coherent job. Anthropic and Microsoft both report that handoffs hurt reliability for tightly-coupled work (e.g., coding), and multi-agent burns ~15× the tokens of a chat. Default to a single ReAct agent with good tools; escalate to multi-agent only for genuine parallelism, isolation, or distinct specialties. See Claude: When to use multi-agent systems and ByteByteGo: How Anthropic Built a Multi-Agent Research System.

3. Supervisor vs. swarm — compare. [medium] Supervisor: central boss delegates to workers with private workspaces; only final outputs are shared; deterministic and testable. Swarm: no boss — the current agent handles or hands off to a peer via Command, no return trip; more flexible, harder to bound and test. See DEV: Supervisor vs Swarm in LangGraph.

4. "Agent-as-tool" — what is it and why is it nice? [medium] Wrap each worker agent as a callable tool on the orchestrator. The orchestrator is then just a ReAct agent whose tools happen to be agents; the framework handles the loop, and you can add a specialist by adding one tool. The concierge does exactly this. Tradeoff: nested agents mean nested token cost and harder tracing.

5. Why did Anthropic's multi-agent research system beat a single agent, and when does that logic fail? [hard] Research questions decompose into independent sub-questions that don't modify shared state, so parallel sub-agents explore a wider search space — ~90% better on their internal eval. It fails for tightly interdependent tasks (coding) where sub-agents would need constant coordination and handoffs corrupt context. See ZenML: Anthropic multi-agent research system.

6. When is the debate pattern worth its cost? [hard] For hard reasoning where a single pass hallucinates or commits early. Multiple agents argue different positions and a judge selects/synthesizes; adversarial critique exposes weak reasoning. Cost scales with rounds × debaters, so reserve it for high-stakes questions. See muthuspark/multi-agent-debate.

7. Explain evaluator–optimizer and give a production example. [medium] A generator produces output, an evaluator scores it against explicit criteria and returns feedback, the generator revises, looping to "pass" or a cap. Production cousins: a grading pipeline's self-consistency confidence gate and the clinical RAG system's schema-validated consensus extraction (generate → validate → retry). See Anthropic: Building Effective Agents.

8. How do you keep a long agent chain reliable? [hard] Because success probability compounds (pⁿ), add checkpoints/retries, per-step structured error capture, confidence gates that branch to human review, and circuit breakers that abort a batch when the early error rate exceeds a threshold — all patterns a robust grading pipeline uses. See Databricks: Agent system design patterns.

---

When to use / tradeoffs

The senior rule of thumb: start single-agent. Add agents only when you can name the specific thing multi-agent buys you — parallelism, context isolation, or a genuinely distinct specialty — and you're willing to pay the token/latency/coordination bill.

---

Summary + related articles

• Memorize the seven patterns and the signal that selects each; interviewers probe the why, not the diagram.
• The dominant tradeoff is coordination cost (tokens ~15×, latency, failure modes) vs. the benefit (parallelism, isolation, specialization).
• Default to a single ReAct agent; escalate deliberately.
• Real systems mix patterns: a concierge orchestrator (orchestrator-worker), a clinical RAG system (parallel subset-of-N), a grading pipeline (sequential + parallel batch + evaluator-optimizer gates).

Related: Agent Orchestration · Agent Communication · Debugging & Observability for Agents

Sources: TrueFoundry · Kore.ai orchestration patterns · Claude: when to use multi-agent · ByteByteGo on Anthropic's system · LangChain: how & when to build multi-agent · Anthropic: Building Effective Agents