Skip to content

IT: allow out-of-order in kill_replica - #1217

Merged
dorjesinpo merged 1 commit into
mainfrom
fix/out-of-order-when-kill-replica
Mar 25, 2026
Merged

IT: allow out-of-order in kill_replica#1217
dorjesinpo merged 1 commit into
mainfrom
fix/out-of-order-when-kill-replica

Conversation

@dorjesinpo

Copy link
Copy Markdown
Collaborator

kill replica can result in out-of-order in the following scenario:

  1. consumer -> proxy -> replica1 -> primary.
  2. kill replica1
  3. proxy detects replica1 disconnect before primary does, and opens the queue on replica2
  4. replica2 opens the queue on primary before primary detects replica1 disconnect, resulting in primary having 2 downstreams.
  5. primary round robins messages {m1, m2}. m1 goes to replica1, m2 goes to replica2
  6. primary detects replica1 disconnect and redelivers m1 to replica2 resulting in out-of-order {m2, m1}

@dorjesinpo
dorjesinpo requested a review from a team as a code owner March 24, 2026 21:11
@dorjesinpo dorjesinpo added the bug Something isn't working label Mar 24, 2026
@dorjesinpo
dorjesinpo requested review from 678098 and removed request for a team March 24, 2026 21:11
@dorjesinpo dorjesinpo changed the title allow out-ot-order in kill_replica allow out-of-order in kill_replica Mar 25, 2026
@dorjesinpo
dorjesinpo force-pushed the fix/out-of-order-when-kill-replica branch 2 times, most recently from 27f64c9 to a4d9d8d Compare March 25, 2026 18:37
@678098 678098 changed the title allow out-of-order in kill_replica IT: allow out-of-order in kill_replica Mar 25, 2026
678098
678098 previously approved these changes Mar 25, 2026
@dorjesinpo
dorjesinpo force-pushed the fix/out-of-order-when-kill-replica branch from a4d9d8d to f8a6d7e Compare March 25, 2026 19:07
Signed-off-by: dorjesinpo <129227380+dorjesinpo@users.noreply.github.com>
@dorjesinpo
dorjesinpo force-pushed the fix/out-of-order-when-kill-replica branch from f8a6d7e to 17a7fcd Compare March 25, 2026 19:11
@dorjesinpo

Copy link
Copy Markdown
Collaborator Author

assume IT failed because of missing sync point. New PR will enforce mqbnet::Channel flush on shutdown

@dorjesinpo
dorjesinpo merged commit c3f4ab7 into main Mar 25, 2026
45 of 48 checks passed
@dorjesinpo
dorjesinpo deleted the fix/out-of-order-when-kill-replica branch March 25, 2026 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants