feat(mls/api): XIP-83 Subscribe — tag deliveries with wave mutate_id and enforce ordering guarantees

[tylerhawkes]

Implements XIP-83 server-tagged deliveries on the v3 Subscribe handler. Spec revision: xmtp/XIPs#139; protos: xmtp/proto#340.

Protocol behavior

• Every Messages frame is stamped with the wave that produced it (mutate_id, 0 = live); a frame never mixes lanes or waves.
• Live frames deliver in ascending id order per message kind, stream-wide; a wave's replay is one merged id-ordered pass across all its topics, per kind.
• The seam: no live frame for a wave's topic before that wave's CatchupComplete; live for other topics keeps flowing; messages are delivered exactly once across the seam.
• A Mutate with adds must carry a nonzero mutate_id (INVALID_ARGUMENT otherwise); every Mutate is acked by exactly one CatchupComplete (immediate when no wave starts).

Implementation

• Chunked per-topic catch-up is replaced by a per-wave merged keyset scan per kind (QueryGroupMessagesWaveScan / QueryWelcomeMessagesWaveScan: unnest-join per-topic floors, global scan cursor, ceiling pinned at wave start, ORDER BY id LIMIT page).
• Live arrivals for a wave's topics are gated into per-topic pending buffers and folded into the wave at completion — sorted per kind, tagged, deduped against what the scan delivered — then TopicsLive, then CatchupComplete, then the gates open.
• Welcome wave scans advance the cursor from raw rows, so a row with an unknown message_type can no longer silently truncate a wave's replay.
• Mutate processing parses and validates all adds before applying removes.

Testing

~20 new tests cover the interleavings: tagging under overlapping waves, merged wave order (both kinds), live total order per kind, the seam (made deterministic via fake-store gating), reset and remove mid-wave, history_only mid-replay publishes, multi-page tag stamping, disconnect cleanup, and store-level unknown-type paging. The suite was mutation-tested: breaking the fold tag, fold sort (either kind), seam gate, fold dedup, welcome cursor advancement, or waveless acks each fails at least one test.

Depends on xmtp/proto#340 — the vendored generated code here matches that branch and regenerates identically from proto@main once it lands.

🤖 Generated with Claude Code

Note

Tag MLS Subscribe delivery frames with wave mutate_id and enforce catch-up ordering guarantees

• Implements XIP-83: all SubscribeResponse_V1_Messages frames now carry a mutate_id correlating them to the originating Mutate wave; live frames are tagged with 0.
• Replaces per-topic batch catch-up with a single ascending-id wave scan per kind (QueryGroupMessagesWaveScan, QueryWelcomeMessagesWaveScan), pinned to a ceiling captured at wave start and paged by a configurable scanPageLimit.
• Live messages arriving mid-wave are buffered (gated), then flushed in ascending-id order via a new completeWave helper before TopicsLive and CatchupComplete are emitted, ensuring no live frames precede the wave seam.
• Adds validation on Mutate requests: nonzero mutate_id required when adds are present; mutate_id reuse while a wave is in-flight is rejected with InvalidArgument; adds-per-Mutate are capped by maxMutateAdds (returns ResourceExhausted).
• Behavioral Change: QueryWelcomeMessagesWaveScan now returns raw scan position metadata (lastRawID, rawCount) and advances the cursor via raw row count to avoid truncation when unknown message_type rows are skipped.

^{Macroscope summarized 6a42e51.}

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR implements XIP-83 Subscribe protocol features including wave mutate_id tagging, new ordering guarantees, and new validation rules that can fail streams. These are significant runtime behavior changes to the subscription system, and the modified files are owned by xmtp/backend team rather than the PR author.

^{You can customize Macroscope's approvability policy. Learn more.}