Summary
Reading a nested subgraph's state returns every DeltaChannel empty, silently. Non-delta channels in the same namespace hydrate correctly, so nothing errors and nothing looks wrong — the caller just sees an empty channel and cannot tell it from a subgraph that genuinely never wrote one.
This bites anyone inspecting run history per subgraph: messages on an agent that opts into delta storage is exactly the channel you want, and it always comes back [].
Repro (self-contained, InMemorySaver)
from typing import Annotated
from typing_extensions import TypedDict
from langgraph.channels.delta import DeltaChannel
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import START, END, StateGraph
def _append(state, writes):
out = list(state or [])
for w in writes:
out.extend(w if isinstance(w, list) else [w])
return out
class S(TypedDict, total=False):
msgs: Annotated[list, DeltaChannel(_append)]
child = StateGraph(S)
child.add_node("a", lambda s: {"msgs": ["a1"]})
child.add_node("b", lambda s: {"msgs": ["b1", "b2"]})
child.add_edge(START, "a"); child.add_edge("a", "b"); child.add_edge("b", END)
parent = StateGraph(S)
parent.add_node("child", child.compile())
parent.add_edge(START, "child"); parent.add_edge("child", END)
app = parent.compile(checkpointer=InMemorySaver())
cfg = {"configurable": {"thread_id": "t1"}}
app.invoke({}, cfg)
child_ns = next(
t.state["configurable"]["checkpoint_ns"]
for snap in app.get_state_history(cfg)
for t in snap.tasks
if t.name == "child" and isinstance(t.state, dict)
)
print(app.get_state(cfg).values["msgs"]) # ['a1', 'b1', 'b2'] ✅
print(app.get_state({"configurable": {"thread_id": "t1",
"checkpoint_ns": child_ns}}).values) # {'msgs': []} ❌
Expected: the child namespace reports ['a1', 'b1', 'b2'].
Actual: [].
Affects get_state, aget_state, get_state_history and aget_state_history.
Cause
A subgraph obtained through get_subgraphs() was compiled without a checkpointer — the parent supplies one via CONFIG_KEY_CHECKPOINTER at read time, which is how the public readers already resolve it (pregel/main.py):
checkpointer = ensure_config(config)[CONF].get(CONFIG_KEY_CHECKPOINTER, self.checkpointer)
But _prepare_state_snapshot then hydrates channels using self.checkpointer alone, so saver is None for any nested subgraph:
channels, managed = channels_from_checkpoint(
self.channels, saved.checkpoint,
saver=self.checkpointer if isinstance(self.checkpointer, BaseCheckpointSaver) else None,
config=saved.config,
)
channels_from_checkpoint needs that saver to call get_delta_channel_history and replay a DeltaChannel's ancestor writes. Without it, the channel falls through to from_checkpoint(MISSING) — empty, no error (pregel/_checkpoint.py):
if delta_channels and saver is not None and config is not None:
histories = saver.get_delta_channel_history(config=config, channels=delta_channels)
...
ch = spec.from_checkpoint(checkpoint["channel_values"].get(k, MISSING))
Recovering it from the config isn't possible either: get_state_history passes checkpoint_tuple.config — the checkpointer's stored config, which never carries CONFIG_KEY_CHECKPOINTER.
Suggested fix
Pass the already-resolved saver into _prepare_state_snapshot / _aprepare_state_snapshot from the four public readers, preferring it over self.checkpointer. +26/−4, read path only.
I have this working with regression tests (subgraph history, subgraph get_state, async history, plus a root-graph control) — the subgraph tests fail on main and pass with the change, and the full suite shows a byte-for-byte identical failure set before and after. Happy to open the PR once this is approved; it was auto-closed for not linking an issue.
How it was found
Building a run-inspection UI on POST /threads/{id}/history. Every nested deep agent reported an empty messages channel while its tasks had demonstrably run many model turns and tool calls — with 1452 kB of message blobs sitting in checkpoint_blobs for one such namespace. Plain agents, whose messages is a BinaryOperatorAggregate rather than a DeltaChannel, were unaffected.
System info
langgraph main @ 1.2.10 (also reproduced on 1.2.1), Python 3.13, macOS.
Summary
Reading a nested subgraph's state returns every
DeltaChannelempty, silently. Non-delta channels in the same namespace hydrate correctly, so nothing errors and nothing looks wrong — the caller just sees an empty channel and cannot tell it from a subgraph that genuinely never wrote one.This bites anyone inspecting run history per subgraph:
messageson an agent that opts into delta storage is exactly the channel you want, and it always comes back[].Repro (self-contained,
InMemorySaver)Expected: the child namespace reports
['a1', 'b1', 'b2'].Actual:
[].Affects
get_state,aget_state,get_state_historyandaget_state_history.Cause
A subgraph obtained through
get_subgraphs()was compiled without a checkpointer — the parent supplies one viaCONFIG_KEY_CHECKPOINTERat read time, which is how the public readers already resolve it (pregel/main.py):But
_prepare_state_snapshotthen hydrates channels usingself.checkpointeralone, sosaverisNonefor any nested subgraph:channels_from_checkpointneeds that saver to callget_delta_channel_historyand replay aDeltaChannel's ancestor writes. Without it, the channel falls through tofrom_checkpoint(MISSING)— empty, no error (pregel/_checkpoint.py):Recovering it from the config isn't possible either:
get_state_historypassescheckpoint_tuple.config— the checkpointer's stored config, which never carriesCONFIG_KEY_CHECKPOINTER.Suggested fix
Pass the already-resolved saver into
_prepare_state_snapshot/_aprepare_state_snapshotfrom the four public readers, preferring it overself.checkpointer. +26/−4, read path only.I have this working with regression tests (subgraph history, subgraph
get_state, async history, plus a root-graph control) — the subgraph tests fail onmainand pass with the change, and the full suite shows a byte-for-byte identical failure set before and after. Happy to open the PR once this is approved; it was auto-closed for not linking an issue.How it was found
Building a run-inspection UI on
POST /threads/{id}/history. Every nested deep agent reported an emptymessageschannel while its tasks had demonstrably run many model turns and tool calls — with 1452 kB of message blobs sitting incheckpoint_blobsfor one such namespace. Plain agents, whosemessagesis aBinaryOperatorAggregaterather than aDeltaChannel, were unaffected.System info
langgraph
main@ 1.2.10 (also reproduced on 1.2.1), Python 3.13, macOS.