Streaming Modes in LangGraph

LangGraphDurable ExecutionAdvanced19 min

By: Anacodic Team

TL;DR

stream_mode chooses what a running graph emits, and the choice is an architecture decision, not a formatting one. "updates" gives you one event per node completion (the diff), "values" gives the whole state after each super-step (the snapshot), "messages" gives LLM tokens as they generate, "custom" carries arbitrary progress you write from inside a node with get_stream_writer(), and "debug" gives everything. You can pass a list of modes and receive (mode, payload) tuples on one stream. The distinction that matters: updates is O(diff) per event while values is O(state) per event, so a graph carrying a large state that streams values re-serialises the entire state on every step — which is how a progress indicator becomes the most expensive part of a run.

Simple explanation + analogy

Think about how you would follow a long train journey.

values is a photograph of the whole train, taken at every station. Complete, self-contained, and you need no history to interpret one — but you carry every passenger's details in every photo, even the ones who never moved.
updates is the station announcement: "three passengers boarded at platform 2." Tiny, and it tells you exactly what changed. To know the current state of the train you have to have heard all the previous announcements.
messages is the tannoy in the carriage, mid-sentence — the words as they are spoken, not a summary afterwards. This is what makes an LLM answer appear token by token.
custom is the guard radioing back whatever they judge worth reporting — "we're 60% through the tunnel." It is not on any timetable and it does not change the train; it exists purely to tell you how things are going.
debug is the full signalling log. You do not read it on a normal day.

The practical consequence is that values and updates are not two views of the same thing with different verbosity. They are snapshots versus diffs, and they have different costs and different failure modes. Snapshots are easy to consume and expensive to send; diffs are cheap to send and require the consumer to keep state.

Diagram

  A three-node graph:  START ─▶ [outline] ─▶ [draft] ─▶ [polish] ─▶ END
                                              │
                                              └─ writer() called 3x inside the node


  stream_mode="updates"   one event per NODE, containing only its diff
  ────────────────────────────────────────────────────────────────────
    {'outline': {'sections': [...]}}          ◀── 3 events total
    {'draft':   {'words': 1800}}
    {'polish':  {'words': 1850}}


  stream_mode="values"    one event per SUPER-STEP, containing ALL state
  ────────────────────────────────────────────────────────────────────
    {topic, sections: [],        words: 0}    ◀── initial state, before any node
    {topic, sections: [i,m,r],   words: 0}         4 events for 3 nodes
    {topic, sections: [i,m,r],   words: 1800}
    {topic, sections: [i,m,r],   words: 1850}
                    ▲
                    └── 'topic' and 'sections' re-sent every time, unchanged


  stream_mode="custom"    only what the node chose to report
  ────────────────────────────────────────────────────────────────────
    {'phase':'draft','section':'intro',  'pct':33}   ◀── from get_stream_writer()
    {'phase':'draft','section':'method', 'pct':67}       never touches state
    {'phase':'draft','section':'results','pct':100}


  stream_mode=["custom","updates"]    interleaved, tagged (mode, payload)
  ────────────────────────────────────────────────────────────────────
    ('updates', {'outline': ...})
    ('custom',  {... 'pct': 33})     ◀── fine-grained progress arrives
    ('custom',  {... 'pct': 67})         DURING the slow node, before
    ('custom',  {... 'pct': 100})        its 'updates' event exists
    ('updates', {'draft': ...})
    ('updates', {'polish': ...})

That last block is the shape most production UIs actually want, and the reason is visible in the ordering: updates cannot tell you anything while a slow node is still running, because the event only exists once the node returns. custom fills exactly that gap.

How it works (deep)

1. `updates` — the diff, one event per node

Each event is {node_name: partial_update}, containing only what that node returned. Node count equals event count, which makes it the natural choice for logging and for driving a step-by-step UI.

It is also what makes a Send fan-out observable: forty parallel branches produce forty updates events even though they all commit in one super-step, so you can count completions for a real progress bar. In values mode the same fan-out gives you one snapshot with all forty results appearing at once and nothing in between.

The cost of the cheapness: the consumer must accumulate. An event tells you words became 1800; it does not tell you what sections currently holds. A client that drops or reorders events drifts out of sync with no way to notice.

2. `values` — the snapshot, one event per super-step

Each event is the entire state after a super-step. Note the count in the example above: four events for three nodes, because the initial state is emitted before any node runs. Off-by-one bugs in progress bars usually trace to this.

Self-contained events mean a late-joining or reconnecting client can render from the most recent one alone. The price is size — the full state on every step, including keys nobody touched.

3. `messages` — LLM tokens

Yields (token_chunk, metadata) as the model generates, with metadata identifying the node and model, so you can route tokens from different nodes to different parts of a UI. This is the mode that produces the typewriter effect users expect from a chat product; without it a response appears only when its node returns.

It requires the node to be using a chat model that supports streaming. A node that calls a model with streaming disabled emits nothing in this mode, which reads as a silent bug — the stream simply produces no messages events.

4. `custom` — progress your state does not store

Inside a node, get_stream_writer() returns a callable. Anything you pass it is emitted immediately:

from langgraph.config import get_stream_writer

def draft(state):
    writer = get_stream_writer()
    for i, s in enumerate(state["sections"], 1):
        writer({"phase": "draft", "section": s, "pct": round(100 * i / len(state["sections"]))})
    return {"words": total}

Two properties make this the most useful mode for real applications. It emits during a node rather than after it, so a node that takes ninety seconds can report progress the whole way. And the payload never enters state, so you are not forced to pollute your state schema with progress_pct fields that no node reads and every checkpoint then persists forever.

The discipline: custom events are advisory. Never make them load-bearing — if a client needs a fact to be correct, it belongs in state where the checkpointer will persist it, because custom payloads are not saved anywhere.

5. `debug` and multiple modes

"debug" emits task scheduling, node starts, and checkpoint writes — verbose, useful when a graph is doing something you cannot explain, not something to ship.

Passing a list changes the yielded shape from payload to (mode, payload):

for mode, payload in app.stream(start, stream_mode=["custom", "updates"]):
    ...

Forgetting that the shape changes when you add a second mode is a routine bug — the loop that worked for one mode now silently unpacks a tuple into the wrong variable.

6. Async, subgraphs, and where events come from

astream() is the async twin with identical modes, and it is what you want behind a web endpoint. For nested graphs, subgraph nodes are invisible by default; subgraphs=True adds a namespace to each event so you can see inside. That is covered in the composition article.

7. When streaming stops being the answer

Streaming keeps a connection open for the duration of a run. For a run measured in minutes-to-days — anything with a human gate — no HTTP connection should be held open that long. There the right shape is: stream while the client is attached, persist state via the checkpointer, and let the client poll or subscribe to reconnect. Streaming is a view onto execution; the checkpointer is the source of truth. Confusing the two produces systems that lose progress whenever a browser tab closes.

The math

The choice between diffs and snapshots is a bandwidth calculation.

  k    = super-steps in the run
  n    = nodes (>= k when fan-out occurs)
  S    = serialised size of full state at the end
  d    = average size of one node's diff

  bytes over the wire:
    updates   B_u = n * d
    values    B_v = sum over i of |state_i|
                  ~ k * S / 2        (state grows roughly linearly to S)
    ratio     B_v / B_u = (k * S) / (2 * n * d)

  event counts:
    updates   = n            (one per node completion)
    values    = k + 1        (one per super-step, PLUS the initial state)
    custom    = however many times you called writer()

Worked example

A 20-step research agent whose final state is S = 200 KB (mostly accumulated messages), average diff d = 4 KB, 20 nodes.

  B_u = 20 * 4 KB                = 80 KB
  B_v = 20 * 200 KB / 2          = 2000 KB  = 2.0 MB
  ratio = 2000 / 80              = 25x

Twenty-five times the traffic to render the same progress. On the three-node example in the code below, with a tiny state, the same arithmetic gives:

  events:  updates = 3        values = 4        custom = 3

which is exactly what the run prints. The event-count asymmetry (3 versus 4) is small here and easy to verify; the byte asymmetry is what scales into a problem.

Real code

A report pipeline where the slow node reports progress that state never stores.

from typing import TypedDict
from langgraph.config import get_stream_writer
from langgraph.graph import StateGraph, START, END

class ReportState(TypedDict):
    topic: str
    sections: list[str]
    words: int

def outline(state: ReportState) -> dict:
    return {"sections": ["intro", "method", "results"]}

def draft(state: ReportState) -> dict:
    writer = get_stream_writer()          # emits progress the state never stores
    total = 0
    for i, s in enumerate(state["sections"], 1):
        total += len(s) * 100
        writer({"phase": "draft", "section": s, "pct": round(100 * i / len(state["sections"]))})
    return {"words": total}

def polish(state: ReportState) -> dict:
    return {"words": state["words"] + 50}

g = StateGraph(ReportState)
for n, f in [("outline", outline), ("draft", draft), ("polish", polish)]:
    g.add_node(n, f)
g.add_edge(START, "outline"); g.add_edge("outline", "draft")
g.add_edge("draft", "polish"); g.add_edge("polish", END)
app = g.compile()

start = {"topic": "grid load", "sections": [], "words": 0}

print("--- updates: what each node changed ---")
for ev in app.stream(start, stream_mode="updates"):
    print(" ", ev)

print("--- values: the whole state after each super-step ---")
for ev in app.stream(start, stream_mode="values"):
    print(" ", ev)

print("--- custom: progress from inside a node ---")
for ev in app.stream(start, stream_mode="custom"):
    print(" ", ev)

print("--- two modes at once: (mode, payload) tuples ---")
for mode, payload in app.stream(start, stream_mode=["custom", "updates"]):
    print(f"  {mode:8s} {payload}")

updates = list(app.stream(start, stream_mode="updates"))
values  = list(app.stream(start, stream_mode="values"))
custom  = list(app.stream(start, stream_mode="custom"))
assert len(updates) == 3, "one event per node"
assert len(values) == 4, "initial state + one per super-step"
assert len(custom) == 3, "one writer() call per section"
assert values[-1]["words"] == 1850, values[-1]["words"]
print(f"updates={len(updates)} values={len(values)} custom={len(custom)} final_words={values[-1]['words']}")

# Output:
#   --- updates: what each node changed ---
#     {'outline': {'sections': ['intro', 'method', 'results']}}
#     {'draft': {'words': 1800}}
#     {'polish': {'words': 1850}}
#   --- values: the whole state after each super-step ---
#     {'topic': 'grid load', 'sections': [], 'words': 0}
#     {'topic': 'grid load', 'sections': ['intro', 'method', 'results'], 'words': 0}
#     {'topic': 'grid load', 'sections': ['intro', 'method', 'results'], 'words': 1800}
#     {'topic': 'grid load', 'sections': ['intro', 'method', 'results'], 'words': 1850}
#   --- custom: progress from inside a node ---
#     {'phase': 'draft', 'section': 'intro', 'pct': 33}
#     {'phase': 'draft', 'section': 'method', 'pct': 67}
#     {'phase': 'draft', 'section': 'results', 'pct': 100}
#   --- two modes at once: (mode, payload) tuples ---
#     updates  {'outline': {'sections': ['intro', 'method', 'results']}}
#     custom   {'phase': 'draft', 'section': 'intro', 'pct': 33}
#     custom   {'phase': 'draft', 'section': 'method', 'pct': 67}
#     custom   {'phase': 'draft', 'section': 'results', 'pct': 100}
#     updates  {'draft': {'words': 1800}}
#     updates  {'polish': {'words': 1850}}
#   updates=3 values=4 custom=3 final_words=1850

Three things to read off the real output. In values mode, 'topic': 'grid load' appears in all four events although no node ever writes it — that is the snapshot overhead, small here and 200 KB per event in a real agent. The values stream has four events for three nodes because the initial state is emitted first. And in the combined stream, all three custom events arrive before ('updates', {'draft': ...}): the progress reports came out of a node that had not yet returned, which is the one thing updates can never do.

Real-world example

A guideline pipeline has a screening stage that filters a few thousand retrieved abstracts down to a shortlist. It is one node and it runs for several minutes.

The first UI streamed values and rendered a spinner between events. Two complaints arrived immediately. Reviewers watching a long screening run had no idea whether it was working or hung, because the node emits nothing until it finishes — values and updates are both blind inside a node. And the page got slower as runs progressed, because state accumulated the shortlist plus per-item scores, so every snapshot re-sent the whole growing structure. The progress display was consuming more bandwidth than the actual results.

The rewrite used stream_mode=["custom", "updates"]. Screening called writer({"screened": i, "total": n, "kept": k}) every fifty abstracts, which gave a genuine moving progress bar during the slow node, and updates marked stage transitions. Traffic dropped by roughly an order of magnitude for the same run because diffs replaced snapshots.

The instructive failure came later. Someone moved a result into a custom payload — the count of screened items the summary page displayed — reasoning that it was already being emitted. It worked in development and produced blanks in production whenever a reviewer reloaded the page mid-run, because custom events are not persisted anywhere: a client that was not connected when the event fired can never obtain it. Results belong in state, where the checkpointer keeps them and any client can read them back. custom is for things that are useful now and worthless later.

Interview questions companies actually ask

1. What is the difference between values and updates? [easy] values emits the entire state after each super-step (a snapshot); updates emits {node: diff} per node completion. Snapshots are self-contained and expensive; diffs are cheap and require the client to accumulate. They also differ in count: values yields one per super-step plus the initial state, while updates yields one per node.

2. Why does values produce four events for a three-node graph? [medium] The initial state is emitted before any node runs, so you get k+1 events for k super-steps. It is the most common off-by-one in progress bars — the first event represents "nothing has happened yet", not "the first node finished".

3. How do you report progress from inside a single long-running node? [medium] get_stream_writer() inside the node, with stream_mode="custom". Neither values nor updates can help, because both only emit when a node returns — a node running for minutes is completely silent in those modes. custom also keeps progress out of the state schema, so checkpoints do not persist it.

4. What changes when you pass a list of modes? [medium] The yielded item becomes (mode, payload) instead of just payload. Adding a second mode to a working single-mode loop breaks it unless you also change the unpacking, which is an easy bug to ship.

5. Why is streaming values on a large-state graph a scaling problem? [hard] Cost per event is O(state), not O(diff), and state usually grows during a run. Total bytes is roughly k*S/2 versus n*d for updates — for a 20-step agent with 200 KB final state and 4 KB diffs, that is 2 MB against 80 KB, about 25x. The progress mechanism ends up dominating the payload.

6. Should a client rely on custom events for correctness? [hard] No. They are advisory and are not persisted, so any client not connected at emission time can never retrieve them — a page reload mid-run loses them permanently. Anything that must be correct belongs in state where the checkpointer holds it. custom is for information that is useful during the run and worthless afterwards.

7. How do you stream tokens from an LLM, and when does that silently fail? [medium] stream_mode="messages", which yields (chunk, metadata) with the node and model identified. It produces nothing if the node's chat model is not configured for streaming — the stream just stays empty in that mode, which looks like a broken graph rather than a misconfigured model call.

8. How does streaming interact with a Send fan-out? [hard] updates gives one event per branch, so forty parallel workers produce forty events and you can count completions. values gives one snapshot in which all forty results appear simultaneously, since the fan-out commits in a single super-step. For per-item progress on a batch, updates is the only one of the two that works.

9. Can you use streaming as the transport for a run with a human approval gate? [hard] Not as the source of truth. A gated run may pause for hours or days, and no HTTP connection should be held open for that. Stream while a client is attached, but treat the checkpointer as authoritative and let clients reconnect and read state. Streaming is a view onto execution, not the record of it.

When to use / tradeoffs

Reach for each mode when:

updates — logging, step-by-step UI, counting completions in a fan-out. The default for most applications.
values — a client that must render from a single event, or reconnect and repaint from the latest snapshot.
messages — token-by-token chat output.
custom — progress from inside a slow node, without polluting the state schema.
debug — local diagnosis of scheduling you cannot explain. Not for production.

Do NOT use when:

Situation	Why it breaks	Use instead
Large or growing state, streaming `values`	O(state) per event; traffic dominated by unchanged keys	`updates`, or `values` throttled
A client needs a fact to be reliable, sent via `custom`	Not persisted; a reconnecting client can never get it	Put it in state
Run may pause for hours (human gate)	No connection should stay open that long	Checkpointer + poll/reconnect
Fine-grained progress, using `updates`	Silent for the whole duration of a slow node	`custom` from inside the node
Verbose `debug` in production	Volume and cost, with no consumer	`updates`

Honest limits. The bandwidth model assumes state grows linearly and diffs are uniform; in real agents both are lumpy, and one node that attaches a large document to state makes a single values event dwarf the entire rest of the run. Streaming also gives you no delivery guarantees — it is a view over execution, not a log, so there is no replay of missed events, no ordering guarantee across a network hiccup, and no way for a client to detect it dropped one. That is precisely why diff-based updates is riskier than it looks for stateful clients: a lost event leaves the consumer silently wrong, whereas a lost values event is corrected by the next one. Choosing updates for cost means accepting that your client's view can drift, and deciding what resynchronises it. And none of the modes tell you a run failed in a structured way; exceptions surface at the stream boundary, so error handling stays the caller's job.

stream_mode picks what is emitted: updates (diffs), values (snapshots), messages (tokens), custom (your progress), debug (everything).
values yields k+1 events for k super-steps — the initial state comes first.
updates is O(diff); values is O(state). On a 20-step, 200 KB-state agent that is ~25x more traffic.
Only custom can report during a node; the others are silent until a node returns.
custom payloads are not persisted — advisory only, never load-bearing.
A list of modes changes the yield to (mode, payload).
Streaming is a view onto execution; the checkpointer is the source of truth for anything that must survive a disconnect.

Related articles in this module (6-6):

Checkpointers, Threads, and Durable Execution — the durable record that streaming is only a view of.
Dynamic Fan-Out with Send — why updates is the mode that makes a parallel batch observable.
Subgraphs and Graph Composition — subgraphs=True and namespaced events from nested graphs.
Time Travel and State History — reconstructing a run after the fact instead of watching it live.

Related elsewhere:

LangGraph — super-steps and the state model these events describe.
Streaming Progress for Long-Running Jobs: Named Events, Interrupts, and Reconnection — the transport-level view: SSE, reconnection, and progress protocols.
Debugging & Observability for Agents — using debug and updates to find where a run went wrong.

Sources: LangGraph docs: streaming · LangGraph how-to: stream outputs · LangGraph reference: get_stream_writer · Verified against langgraph 1.2.8; all output above is from an actual run.

← Previous: Time Travel and State History Next: Dynamic Fan-Out with Send →