Dynamic Fan-Out with Send

TL;DR

Send lets one node dispatch N parallel copies of another node when N is only known at runtime. A normal edge is drawn once at build time, so it can express "after A, run B" but never "after A, run B once per item in a list whose length I discover mid-run". Send("worker", payload) decouples which node runs from what state it sees — you return a list of Send objects from a conditional edge, and LangGraph schedules every one of them in a single super-step, merging their results through a reducer. The whole pattern collapses if the target key has no reducer: parallel writes to the same un-reduced key raise InvalidUpdateError, or silently keep only one result. It also stops being the right tool the moment your items are not independent — Send gives you map-reduce, and map-reduce assumes the map steps don't need to talk to each other.

---

Simple explanation + analogy

A normal LangGraph edge is a printed org chart: it says "requests go from intake to review", and that arrow is fixed the day you print it. It cannot say "route this to one reviewer per application" because when you printed the chart you didn't know how many applications would arrive.

Send is the manager handing out copies at the morning meeting. Forty applications came in overnight; the manager makes forty envelopes, drops one on each of forty desks, and everyone works at once. There is still only one "reviewer" job description on the org chart — but this morning it is being performed forty times in parallel, each instance seeing only its own envelope.

Two things make this work, and both matter:

• Each worker gets its own payload, not the whole state. The envelope contains one application, not the stack of forty. This is the part people find surprising: a Send target receives the payload you passed, not the graph state its siblings can see.
• The results have to be merged, not overwritten. Forty reviewers writing to one clipboard field would clobber each other. A reducer on that field turns forty writes into one concatenated list.

Take either piece away and the pattern breaks. Miss the payload semantics and your worker reads state["applications"] and re-does the whole batch, forty times. Miss the reducer and thirty-nine results vanish.

---

Diagram

  WITHOUT Send — the graph shape is fixed at build time
  ----------------------------------------------------
        START ──▶ [score_batch] ──▶ [shortlist] ──▶ END
                       │
                       └─ loops over 4 items INSIDE one node:
                          4 sequential LLM calls, 1 super-step,
                          one slow node you cannot retry per-item

  WITH Send — the fan is opened at run time
  ----------------------------------------------------
        START
          │
          └─ fan_out(state) returns [Send, Send, Send, Send]
                 │        │        │        │
                 ▼        ▼        ▼        ▼
           [score_one][score_one][score_one][score_one]   ◀── ONE declared node,
             ana        bo        cyra       dev              four invocations
                 │        │        │        │
                 └────────┴───┬────┴────────┘
                              ▼   reducer: operator.add merges 4 lists into 1
                        [shortlist]
                              │
                              ▼
                             END

  super-steps:  START ──▶ (all four scorers) ──▶ shortlist
                  0              1                   2
                              ^^^^^^^
                    all four resolve in a SINGLE super-step

The right-hand shape is what Send buys you: the node count in your source stays at one, while the invocation count scales with the data.

---

How it works (deep)

1. Send is a value you return, not a function you call

Send(node_name, payload) is an inert dataclass. Returning it from a routing function tells LangGraph: schedule node_name, and when you run it, hand it payload as its input instead of the graph state.

from langgraph.types import Send

def fan_out(state):
    return [Send("score_one", {"application": a}) for a in state["applications"]]

You attach it with add_conditional_edges, because fanning out is a routing decision — it just happens to be a routing decision with multiplicity:

g.add_conditional_edges(START, fan_out, ["score_one"])

The third argument is the list of nodes this router may reach. At runtime LangGraph does not need it; it is there so the drawn graph and any static validation know score_one is reachable from START.

2. The payload replaces the state, and that is the whole point

Inside score_one, the parameter is the dict you passed to Send — {"application": {...}}. It is not the graph state. There is no state["applications"] to read.

This is deliberate. If every parallel worker saw the full state, you would have forty workers each capable of processing all forty items, and nothing to tell worker 7 that it owns item 7. Passing an explicit payload is what makes the branches disjoint.

The practical consequence: a Send target usually needs a different type signature from a normal node. A normal node is State -> partial State. A Send target is Payload -> partial State. It reads a payload and writes to graph state — asymmetric, and worth a comment in your code because it reads oddly the first time.

3. Reducers are mandatory, not optional

Every Send branch returns a partial state update, and they all land at the same moment. For a key without a reducer, LangGraph's default merge is overwrite — and four concurrent overwrites of one key in one super-step is exactly the conflict it refuses to guess at.

class ScreenState(TypedDict):
    scored: Annotated[list[dict], operator.add]   # concatenate

operator.add on a list is concatenation, so four {"scored": [one_result]} updates become one four-element list. add_messages from langgraph.graph.message is the same idea specialised for chat history (it also de-duplicates by message id). For counters, operator.add on an int sums them.

Choose the reducer for the merge semantics you want, not for the type. A reducer that concatenates gives you all results in nondeterministic order; if you need them ordered, carry the index in the payload and sort afterwards — do not assume the fan-in preserves your dispatch order, because it does not.

4. One super-step, and what that means for failure

LangGraph executes in super-steps: it runs every scheduled node, then commits all their state updates atomically, then decides what to schedule next. A fan-out of forty Sends is one super-step containing forty node executions.

Atomic commit is the part with teeth. If one of the forty raises and you have no error handling, the super-step fails and none of the forty updates are committed — you lose the thirty-nine that succeeded. Two ways out:

• Catch inside the worker and return a partial result carrying the error, e.g. {"scored": [{"name": n, "error": str(e)}]}. The super-step then always succeeds and the fan-in node decides what to do with failures. This is almost always what you want for a batch.
• Let it fail and rely on the checkpointer to resume the whole super-step. Correct, but it re-runs all forty, so it only makes sense when the work is cheap or genuinely transactional.

5. Where the parallelism actually comes from

Send gives you concurrency, and whether that becomes speed depends on the work. For I/O-bound nodes — an LLM call, a retrieval, an HTTP request — use async def workers and ainvoke, and the forty calls genuinely overlap. For CPU-bound Python work, the GIL means forty Sends buy you scheduling clarity and per-item retryability but very little wall-clock time.

Bound the width. Forty concurrent LLM calls will hit a provider rate limit, and LangGraph will happily dispatch all forty. Cap it with a semaphore inside the worker, or chunk the payloads so each Send handles a slice of items instead of one.

6. When this stops applying

Send assumes the branches are independent. If worker 7's result changes what worker 8 should do, you do not have a map-reduce — you have a sequence, and forcing it into a fan-out will produce results that depend on scheduling order. Use a loop with a conditional edge instead.

---

The math

Fan-out is the classic parallel-speedup calculation, with a ceiling imposed by your concurrency cap.

  N  = number of items (known only at run time)
  t  = time for one item
  P  = max concurrent workers (rate limit, pool size, semaphore)

  sequential loop inside one node:   T_seq = N * t
  Send fan-out:                      T_fan = ceil(N / P) * t
  speedup:                           S     = N / ceil(N / P)

Super-step accounting is what distinguishes the two shapes:

  loop-inside-a-node:  super-steps = 1     node executions = 1
  Send fan-out:        super-steps = 1     node executions = N

Both are one super-step. The difference is N executions the runtime can schedule, observe, and stream individually versus one opaque node.

Worked example

A batch of N = 40 applications, each needing one LLM call at t = 2.0 s, provider cap P = 8.

  T_seq = 40 * 2.0                = 80.0 s
  T_fan = ceil(40 / 8) * 2.0
        = 5 * 2.0                 = 10.0 s
  S     = 40 / 5                  = 8.0x

The speedup saturates at P, not at N — going from a cap of 8 to unlimited concurrency does nothing for you unless the provider agrees. And with N = 4, P = 8:

  T_fan = ceil(4 / 8) * 2.0 = 1 * 2.0 = 2.0 s      S = 4 / 1 = 4.0x

Below the cap, speedup equals N. That is the regime the runnable example below sits in — four items, no cap — which is why all four scorers land in one super-step.

---

Real code

Screening a batch of job applications. The batch size is a runtime fact; the graph declares exactly one scorer.

import operator
from typing import Annotated, TypedDict
from langgraph.graph import StateGraph, START, END
from langgraph.types import Send

# One application is scored on its own; the batch size is only known at runtime.
class ScreenState(TypedDict):
    applications: list[dict]
    scored: Annotated[list[dict], operator.add]   # reducer: concatenate, never overwrite
    shortlist: list[str]

REQUIRED = {"python", "sql", "airflow"}

def score_one(payload: dict) -> dict:
    """Stands in for an independent per-item LLM call."""
    app = payload["application"]
    have = REQUIRED & set(app["skills"])
    return {"scored": [{"name": app["name"], "score": round(len(have) / len(REQUIRED), 2)}]}

def fan_out(state: ScreenState):
    # One Send per application: N is read from state, not fixed in the graph.
    return [Send("score_one", {"application": a}) for a in state["applications"]]

def shortlist(state: ScreenState) -> dict:
    return {"shortlist": sorted(s["name"] for s in state["scored"] if s["score"] >= 0.66)}

g = StateGraph(ScreenState)
g.add_node("score_one", score_one)
g.add_node("shortlist", shortlist)
g.add_conditional_edges(START, fan_out, ["score_one"])
g.add_edge("score_one", "shortlist")
g.add_edge("shortlist", END)
app = g.compile()

batch = [
    {"name": "ana",  "skills": ["python", "sql", "airflow"]},
    {"name": "bo",   "skills": ["python", "excel"]},
    {"name": "cyra", "skills": ["python", "sql"]},
    {"name": "dev",  "skills": ["scala"]},
]
start = {"applications": batch, "scored": [], "shortlist": []}
out = app.invoke(start)

for s in sorted(out["scored"], key=lambda x: -x["score"]):
    print(f'  {s["name"]:6s} {s["score"]}')
print("shortlist:", out["shortlist"])

# The graph declares ONE score_one node, yet it ran once per application.
assert len(out["scored"]) == len(batch), "every application produced exactly one score"
assert out["shortlist"] == ["ana", "cyra"], out["shortlist"]

# "updates" yields one event per node completion -> 4 parallel score_one events.
node_events = [k for s in app.stream(start, stream_mode="updates") for k in s]
print("node completion events:", node_events)

# "values" yields one snapshot per SUPER-STEP: the 4 scorers share a single step.
snapshots = list(app.stream(start, stream_mode="values"))
print("super-step snapshots  :", len(snapshots))
print("scored count per step :", [len(s.get("scored", [])) for s in snapshots])

assert node_events.count("score_one") == 4, "one invocation per item"
assert len(snapshots) == 3, "START -> fan-out step -> shortlist step"
print("OK: 1 declared node, 4 parallel invocations, merged in 1 super-step")

# Output:
#     ana    1.0
#     cyra   0.67
#     bo     0.33
#     dev    0.0
#   shortlist: ['ana', 'cyra']
#   node completion events: ['score_one', 'score_one', 'score_one', 'score_one', 'shortlist']
#   super-step snapshots  : 3
#   scored count per step : [0, 4, 4]
#   OK: 1 declared node, 4 parallel invocations, merged in 1 super-step

Read scored count per step : [0, 4, 4] carefully — it is the clearest evidence of the mechanism. The count goes 0 → 4 with no intermediate 1, 2, 3. All four results appear at the same commit, because the fan-out is one atomic super-step, not four sequential ones.

Note also that cyra scored 0.67 and made the shortlist against a 0.66 threshold. That is a one-item margin decided by float rounding, and it is exactly the kind of boundary an assert should pin down — the first version of this example asserted ["ana"] and was wrong.

---

Real-world example

A document-review pipeline retrieves candidate papers for a query, then needs a relevance judgement on each. Retrieval returns however many it returns — 12 for a narrow query, 200 for a broad one.

The first implementation looped inside a single score_all node. It worked, and it had three problems that only showed up under load. A 200-document query took 200 sequential LLM calls and timed out the request. When call 147 failed on a malformed response, the exception killed the node and lost the 146 completed judgements. And the progress bar could only ever show "scoring…" because the runtime saw one node, so there was nothing finer to stream.

Rewriting it as Send fixed all three for the same reason: the runtime could finally see the individual units of work. 200 async calls overlapped up to the provider cap. A try/except inside the worker turned a failed judgement into {"error": ...} in the results list, so the batch completed and the fan-in node reported "197 scored, 3 failed" instead of dying. And stream_mode="updates" emitted one event per document, which the UI counted into a real progress bar.

The bug that cost the most time was none of those. It was forgetting the reducer on the results key: the first Send version returned {"scored": [result]} from each of 200 workers with no Annotated[..., operator.add], and the run finished suspiciously fast with exactly one judgement in the output. Not an error — just 199 silently discarded overwrites. That failure mode is quiet, which is why it is worth an assert on len(results) == len(inputs) in any fan-out you ship.

---

Interview questions companies actually ask

1. Why can't a normal conditional edge express fan-out over a runtime-sized list? [easy] A conditional edge's router returns the name of the next node (or a list of names), and the graph topology is fixed when you compile. It selects among nodes that already exist; it cannot create N invocations of one node, and it has no way to give each invocation a different input. Send adds exactly the missing piece — a (node, payload) pair, returned as many times as you like.

2. What does a Send target receive as its argument? [medium] The payload you constructed, not the graph state. Send("score_one", {"application": a}) means score_one is called with {"application": a}. This is what makes the branches disjoint — each worker can only see its own item. Its return value, however, is a normal partial update against the graph state, so a Send target has an asymmetric signature: payload in, state update out.

3. What happens if two Send branches write the same state key and it has no reducer? [hard] The default merge for a key is overwrite, and concurrent overwrites in one super-step are a conflict. Depending on the key and version you either get an InvalidUpdateError or you get one arbitrary winner and the rest silently dropped. Both are bugs; the silent one is worse. Any key a fan-out writes needs a reducer — operator.add for lists and numbers, add_messages for chat history.

4. How many super-steps does a fan-out of 40 items take? [medium] One. A super-step is "run everything currently scheduled, then commit all updates atomically", so 40 Sends are 40 node executions inside a single super-step. That is why stream_mode="values" shows the results appearing all at once, while stream_mode="updates" shows 40 separate node-completion events.

5. One item in a 40-item fan-out throws. What happens to the other 39? [hard] Their updates are lost, because the super-step commits atomically — a failure anywhere means nothing is committed. With a checkpointer the whole super-step can resume, but that re-runs all 40. The production pattern is to catch the exception inside the worker and return the error as data, so the super-step always commits and the fan-in node handles partial failure explicitly.

6. Does Send make my graph faster? [medium] It makes it concurrent, which becomes faster only for I/O-bound work with async nodes. For CPU-bound Python the GIL means you get little wall-clock benefit — though you still gain per-item observability and retryability. And speedup is capped by your concurrency limit P, not by N: at N=40, P=8 you get 8x, and raising N alone does not improve it.

7. Can the results of a fan-out be relied on to arrive in dispatch order? [hard] No. Completion order is nondeterministic, and a concatenating reducer appends in completion order. If order matters, put the index in the payload and sort in the fan-in node. Assuming input order is a bug that hides until one call happens to be slow.

8. When is Send the wrong tool? [medium] When the items are not independent. Send is map-reduce, and map-reduce assumes no map step needs another's output. If item 8's processing depends on item 7's result, you need a sequential loop with a conditional edge; a fan-out would make the outcome depend on scheduling.

---

When to use / tradeoffs

Reach for Send when:

• The number of parallel branches is a runtime value — retrieval hits, uploaded files, rows in a batch.
• The per-item work is I/O-bound (LLM calls, retrieval, HTTP) and genuinely overlaps.
• You want per-item observability: streaming progress, per-item retries, partial-failure reporting.
• Each item is independent and the results merge cleanly through a reducer.

Do NOT use when:

Honest limits. The speedup model above is optimistic in three ways worth naming. It assumes every item takes the same time t, so a single straggler in a fan-out of 40 holds the entire super-step open — real tail latency is set by your slowest item, not your mean. It assumes concurrency is free, ignoring memory: 200 concurrent branches each holding a document in state is 200 documents resident at once, and state size is what usually breaks first, not CPU. And it ignores that the atomic super-step makes partial failure your problem — the runtime's default behaviour discards successful work, so every fan-out that matters needs explicit in-worker error handling before it is production-ready. Fan-out buys throughput and observability; it does not buy resilience.

---

Summary + related articles

• Send(node, payload) decouples which node runs from what input it sees, which is what lets one declared node run N times with N discovered at runtime.
• A Send target receives the payload, not the graph state — that asymmetry is the mechanism that keeps branches disjoint.
• Every key a fan-out writes needs a reducer; without one, concurrent updates either error or silently drop results.
• The whole fan-out is one super-step with an atomic commit, so one failure discards all sibling results unless you catch errors inside the worker.
• Speedup is N / ceil(N/P), capped by your concurrency limit P — and only for I/O-bound work.
• It stops being the right pattern the moment the items are no longer independent.

Related articles in this module (6-6):

• Streaming Modes in LangGraph — how updates vs values exposes a fan-out's individual branches for a progress bar.
• Command and Dynamic Routing — the other way a node influences control flow, for single-destination handoffs.
• Subgraphs and Graph Composition — fanning out to a compiled subgraph instead of a plain function.
• Checkpointers, Threads, and Durable Execution — what resuming a failed super-step actually re-runs.

Related elsewhere:

• LangGraph — the base model of state, nodes, edges, and reducers this builds on.
• Agent Orchestration — orchestrator-worker patterns and partial-failure handling above the graph level.
• Multi-Agent Patterns — when the parallel workers are whole agents rather than functions.

Sources: LangGraph docs: Send API · LangGraph: map-reduce branches · LangGraph low-level concepts: super-steps and reducers · Verified against langgraph 1.2.8; all output above is from an actual run.