Add mutate_id wave tag to Subscribe delivery frames#340
Merged
Conversation
This was referenced Jul 9, 2026
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
Contributor
ApprovabilityVerdict: Approved Additive proto schema change adding a new You can customize Macroscope's approvability policy. Learn more. |
Messages.mutate_id (mls v3) and Envelopes.mutate_id (xmtpv4) carry the catch-up wave that produced each frame (0 = live tail). Mutate.mutate_id is required nonzero when adds are present; comments document the order guarantees, the catch-up seam, and CatchupComplete acking every Mutate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tylerhawkes
force-pushed
the
tyler/xip-83-subscribe-delivery-tags
branch
from
July 9, 2026 09:13
d1c60ff to
da6dcf1
Compare
tylerhawkes
added a commit
to xmtp/xmtp-node-go
that referenced
this pull request
Jul 10, 2026
…and enforce ordering guarantees (#564) Implements XIP-83 server-tagged deliveries on the v3 `Subscribe` handler. Spec revision: xmtp/XIPs#139; protos: xmtp/proto#340. ## Protocol behavior - Every `Messages` frame is stamped with the wave that produced it (`mutate_id`, `0` = live); a frame never mixes lanes or waves. - Live frames deliver in ascending id order per message kind, stream-wide; a wave's replay is one merged id-ordered pass across all its topics, per kind. - The seam: no live frame for a wave's topic before that wave's `CatchupComplete`; live for other topics keeps flowing; messages are delivered exactly once across the seam. - A `Mutate` with adds must carry a nonzero `mutate_id` (`INVALID_ARGUMENT` otherwise); every `Mutate` is acked by exactly one `CatchupComplete` (immediate when no wave starts). ## Implementation - Chunked per-topic catch-up is replaced by a per-wave merged keyset scan per kind (`QueryGroupMessagesWaveScan` / `QueryWelcomeMessagesWaveScan`: unnest-join per-topic floors, global scan cursor, ceiling pinned at wave start, `ORDER BY id LIMIT page`). - Live arrivals for a wave's topics are gated into per-topic pending buffers and folded into the wave at completion — sorted per kind, tagged, deduped against what the scan delivered — then `TopicsLive`, then `CatchupComplete`, then the gates open. - Welcome wave scans advance the cursor from raw rows, so a row with an unknown `message_type` can no longer silently truncate a wave's replay. - `Mutate` processing parses and validates all adds before applying removes. Depends on xmtp/proto#340 — the vendored generated code here matches that branch and regenerates identically from proto@main once it lands. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
tylerhawkes
added a commit
to xmtp/xmtpd
that referenced
this pull request
Jul 14, 2026
…enforce ordering guarantees (#2035) Implements XIP-83 server-tagged deliveries on the d14n `QueryApi.Subscribe` handler. Spec revision: xmtp/XIPs#139; protos: xmtp/proto#340. ## Protocol behavior - Every `Envelopes` frame is stamped with the wave that produced it (`mutate_id`, `0` = live); a frame never mixes lanes or waves. - Live frames deliver each originator's envelopes in ascending `originator_sequence_id`, stream-wide; a wave's replay does the same across the wave's topics. - The seam: no live frame for a wave's topic before that wave's `CatchupComplete`; live for other topics keeps flowing; envelopes are delivered exactly once across the seam. - A `Mutate` with adds must carry a nonzero `mutate_id` (`INVALID_ARGUMENT` otherwise); every `Mutate` is acked by exactly one `CatchupComplete`. ## Implementation - Per-topic chunked catch-up is replaced by one merged keyset scan (`SelectGatewayEnvelopesWaveScan`: per-(topic, originator) floors, row-value `(originator, sequence)` keyset progression, per-originator ceilings from the new `SelectOriginatorCeilings` pinned at wave start). - Live arrivals for a wave's topics are gated and folded into the wave at completion (sorted per originator, tagged, deduped via the topic cursor), then `TopicsLive`, then `CatchupComplete`, then the gates open. - The scan position advances from raw rows; page fetches retry with backoff and send only after a fully successful fetch. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the XIP-83 server-side delivery tag to both
Subscribebindings, per the revised spec (xmtp/XIPs#139):mls.api.v1 SubscribeResponse.V1.Messages.mutate_id = 3andxmtpv4.message_api SubscribeResponse.V1.Envelopes.mutate_id = 2: every delivery frame carries the catch-up wave that produced it (the originatingMutate'smutate_id);0means live tail. A frame never mixes lanes or waves.Mutate.mutate_idis required nonzero whenaddsare present (the server fails the stream withINVALID_ARGUMENTotherwise) and SHOULD be unique per stream.CatchupComplete),CatchupCompleteacking everyMutate(immediately when no wave starts),history_onlymeaning "everything as of the wave's start", and removes clearing the topic's cursor floor.Wire-compatible:
buf breakingagainstmainis clean (additive fields + comment changes only).Landing order: this PR lands first; the xmtp-node-go and xmtpd implementation PRs vendor code generated from these protos and follow.
🤖 Generated with Claude Code
Note
Add
mutate_idfield to Subscribe delivery frames for catch-up wave trackingmutate_id(uint64) toSubscribeResponse.V1.Messagesin mls.proto andSubscribeResponse.V1.Envelopesin message_api.proto; nonzero values identify catch-up replay frames belonging to a specific wave, while 0 indicates live tail frames.CatchupCompletecomments to clarify it serves as the seam after which live (mutate_id 0) frames begin for the wave's topics.mutate_idvalidation rules onSubscribeRequest.V1.Mutate: must be nonzero when adds are present and must not collide with an in-flight wave; violations fail the stream withINVALID_ARGUMENT.Macroscope summarized da6dcf1.