Checked other resources
Related Issues / PRs
#8382 and #8384 are also DeltaChannel bugs but unrelated to this one (write ordering within a superstep, and the InMemorySaver migration override).
Reproduction Steps / Example Code (Python)
import asyncio
from typing import Annotated, TypedDict
from langgraph.channels.delta import DeltaChannel
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import StateGraph
def append(current, writes):
out = list(current or [])
for w in writes:
out.extend(w if isinstance(w, list) else [w])
return out
class S(TypedDict):
log: Annotated[list, DeltaChannel(append, snapshot_frequency=1000)]
def build(cp, tag):
async def node(state):
return {"log": [f"{tag}-out"]}
b = StateGraph(S)
b.add_node("n", node)
b.set_entry_point("n")
b.set_finish_point("n")
return b.compile(checkpointer=cp)
async def main():
cp = InMemorySaver()
cfg = {"configurable": {"thread_id": "t"}}
await build(cp, "first").ainvoke({"log": ["in-1"]}, cfg)
g = build(cp, "second")
await g.ainvoke({"log": ["in-2"]}, cfg)
# the checkpoint before "in-2" entered state
base = None
async for snap in g.aget_state_history(cfg):
if "in-2" not in snap.values["log"]:
base = snap
break
print("fork base:", base.values["log"])
fork_cfg = {
"configurable": {
"thread_id": "t",
"checkpoint_ns": "",
"checkpoint_id": base.config["configurable"]["checkpoint_id"],
}
}
await build(cp, "third").ainvoke({"log": ["in-3"]}, fork_cfg)
print("actual :", (await g.aget_state(cfg)).values["log"])
print("expected:", base.values["log"] + ["in-3", "third-out"])
asyncio.run(main())
Output:
fork base: ['in-1', 'first-out']
actual : ['in-1', 'first-out', 'in-2', 'in-3', 'third-out']
expected: ['in-1', 'first-out', 'in-3', 'third-out']
'in-2' belongs to the branch that was abandoned by forking, and it should not be in the fork's state.
update_state against an older checkpoint has the same problem — replacing the last two lines with
forked = await g.aupdate_state(fork_cfg, {"log": ["patched"]})
print((await g.aget_state(forked)).values["log"])
gives ['in-1', 'x-out', 'patched', 'in-2'] instead of ['in-1', 'x-out', 'patched'].
Description
Addressing an older checkpoint creates a fork: the shared parent ends up with two children. get_delta_channel_history walks the parent chain and collects every pending_writes entry it finds on each on-path ancestor — but the shared parent also holds the writes of the sibling branch, and nothing in the stored data records which child consumed which write. So the abandoned branch's writes are replayed into the fork.
The method's docstring says:
Walks the parent chain (not list(before=...)): for forked threads, only on-path ancestors contribute.
The ancestors it selects are indeed on-path. The writes hanging off them are not.
This is not specific to one saver: InMemorySaver, the base walk (used by SQLite), and PostgresSaver's own paged-SQL override all reproduce it, which suggests write-to-child ownership is a gap in the delta contract rather than one implementation's slip. full (non-delta) channels are unaffected, since their checkpoints carry complete channel_values and need no replay.
I hit this in a chat application where "regenerate the last answer" resumes from the checkpoint before that turn: the superseded answer came back alongside the new one after a reload.
Proposed fix — write ownership is not recoverable at read time, so make the fork's first checkpoint self-contained instead: force delta channels to snapshot when a run is entered against an explicitly addressed checkpoint, and when update_state writes against one. The walk then terminates at the fork instead of reaching the shared parent. This mirrors the existing force-snapshot for Overwrite writes, which exists for the same reason. Cost is one extra snapshot per explicitly addressed resume — not per superstep.
I have this implemented and tested against main (_loop.py, _checkpoint.py, main.py, plus a regression test file; make format / make lint clean, and the full libs/langgraph suite shows no new failures versus unpatched main). Happy to open the PR — could a maintainer assign me to this issue?
System Info
langgraph 1.2.9 (and reproduced on main @ 30c4d58), langgraph-checkpoint 4.1.1, langgraph-checkpoint-sqlite 3.1.0
Python 3.11.5, macOS 26.4 arm64
Checked other resources
Related Issues / PRs
#8382 and #8384 are also
DeltaChannelbugs but unrelated to this one (write ordering within a superstep, and theInMemorySavermigration override).Reproduction Steps / Example Code (Python)
Output:
'in-2'belongs to the branch that was abandoned by forking, and it should not be in the fork's state.update_stateagainst an older checkpoint has the same problem — replacing the last two lines withgives
['in-1', 'x-out', 'patched', 'in-2']instead of['in-1', 'x-out', 'patched'].Description
Addressing an older checkpoint creates a fork: the shared parent ends up with two children.
get_delta_channel_historywalks the parent chain and collects everypending_writesentry it finds on each on-path ancestor — but the shared parent also holds the writes of the sibling branch, and nothing in the stored data records which child consumed which write. So the abandoned branch's writes are replayed into the fork.The method's docstring says:
The ancestors it selects are indeed on-path. The writes hanging off them are not.
This is not specific to one saver:
InMemorySaver, the base walk (used by SQLite), andPostgresSaver's own paged-SQL override all reproduce it, which suggests write-to-child ownership is a gap in the delta contract rather than one implementation's slip.full(non-delta) channels are unaffected, since their checkpoints carry completechannel_valuesand need no replay.I hit this in a chat application where "regenerate the last answer" resumes from the checkpoint before that turn: the superseded answer came back alongside the new one after a reload.
Proposed fix — write ownership is not recoverable at read time, so make the fork's first checkpoint self-contained instead: force delta channels to snapshot when a run is entered against an explicitly addressed checkpoint, and when
update_statewrites against one. The walk then terminates at the fork instead of reaching the shared parent. This mirrors the existing force-snapshot forOverwritewrites, which exists for the same reason. Cost is one extra snapshot per explicitly addressed resume — not per superstep.I have this implemented and tested against
main(_loop.py,_checkpoint.py,main.py, plus a regression test file;make format/make lintclean, and the fulllibs/langgraphsuite shows no new failures versus unpatchedmain). Happy to open the PR — could a maintainer assign me to this issue?System Info
langgraph 1.2.9 (and reproduced on
main@ 30c4d58), langgraph-checkpoint 4.1.1, langgraph-checkpoint-sqlite 3.1.0Python 3.11.5, macOS 26.4 arm64