Skip to content

[feat] Bucket layer for sync#394

Merged
St4NNi merged 97 commits into
mainfrom
feat/bucket-sync
Jul 13, 2026
Merged

[feat] Bucket layer for sync#394
St4NNi merged 97 commits into
mainfrom
feat/bucket-sync

Conversation

@St4NNi

@St4NNi St4NNi commented Jul 11, 2026

Copy link
Copy Markdown
Member

Closes #262

Adds the shard-based sync partition layer on top of placement.

  • Documents are grouped into shards (the sync partition unit; "shard" avoids collision with S3 buckets), each with a per-(strategy, shard) topic, manifest, and digest-based anti-entropy.
  • Shard-topic genesis is single-origin by rank uniqueness: only the resolver's rank-0 holder of a (strategy, epoch, shard) creates the topic genesis; other nodes join-only. Outbox records whose shard-topic genesis hasn't arrived are deferred until it does, avoiding split-brain from concurrent first-writers.
  • Bucket assignment and topic derivation are wired through document sync, onboarding, metadata, and the admin outbox; node-info and onboarding-placement invariants from the placement core are preserved.

Depends on #261 (merged).

St4NNi added 30 commits July 6, 2026 20:53
@St4NNi St4NNi merged commit 7133e70 into main Jul 13, 2026
12 checks passed
@St4NNi St4NNi deleted the feat/bucket-sync branch July 13, 2026 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feat] Placement-bucket manifests and irokle topic layer

1 participant