@@ -61,9 +61,20 @@ when the cluster is at its intended size.
6161Two properties fall out of this rather than needing rules of their own:
6262
6363- ** Claims are atomic, so two replicas can never hold the same index.** The settle window is therefore never a
64- correctness concern, however the timing falls.
65- - ** Coverage is always complete.** An index nobody claimed is picked up by the replica below it, so running fewer
66- replicas than intended means somebody works harder, not that a guild stops being watched.
64+ correctness concern, however the timing falls. This one holds at every instant.
65+ - ** Coverage converges.** An index nobody claimed gets picked up by the replica below it, so running fewer
66+ replicas than intended costs effort rather than coverage — but that is the _ settled_ state, not a continuous
67+ guarantee. A replica that dies takes its shards down until something reclaims its index, so ** transient dark
68+ shards are possible and expected** :
69+
70+ | How the index came free | Dark for |
71+ | ---------------------------------------- | ------------------------------------------------------------------------------------------------ |
72+ | Crash or planned restart | that container's restart, typically seconds — it reclaims its own index |
73+ | Removed for good (scale-down, host gone) | up to the lease TTL, plus two watcher checks, plus the neighbour's restart — on the order of 90s |
74+
75+ Nothing shortens the second case without giving up the debounce that stops a peer merely between renewals from
76+ bouncing the cluster. If a bot ever needs sub-minute recovery from a permanently removed replica, that is a
77+ reason to revisit the watcher interval, not the assignment scheme.
6778
6879An index owns a slice that is the same shards no matter how many peers are up. That is what lets a replica hold
6980its assignment for its whole lifetime instead of resharding underneath itself whenever the cluster changes size.
@@ -77,17 +88,17 @@ instead of `4/4/4/2` for 14 shards) and was rejected anyway, because it trades t
7788- ** An index must mean the same shards regardless of the current shard count.** Flat slices survive Discord
7889 raising its recommendation: 14 → 15 grows only the tail replica. Proportional boundaries move on almost any
7990 bump, so every replica reshards, every replica restarts, and every replica's guilds change.
80- - ** ` SHARDS_PER_REPLICA ` stays the capacity bound its name promises ** , not a divisor. A replica never exceeds it
81- unless it is covering for a peer that never claimed its index .
91+ - ** ` SHARDS_PER_REPLICA ` stays a shard count ** , not a divisor — index ` i ` means the same slice whatever the
92+ cluster is doing, which is what makes the previous point true .
8293
8394So a short tail is ** headroom, not imbalance** — ` 4/4/4/2 ` fills to ` 4/4/4/3 ` , ` 4/4/4/4 ` , and then a fifth index
84- appears. Sizing a container is a question about ` SHARDS_PER_REPLICA ` alone.
95+ appears.
8596
8697Genuine imbalance has exactly two causes, and both are the cluster not being at its intended size:
8798
8899- ** A replica is missing.** A peer covers its indices. This is unavoidable rather than a design choice: if three
89- replicas must cover four indices' worth of shards, one of them holds more. The alternative is dark shards, which
90- is the trade this design deliberately refuses.
100+ replicas must cover four indices' worth of shards, one of them holds more. The alternative is leaving those
101+ shards uncovered for as long as the replica stays away, which is the trade this design refuses.
91102- ** A straggler.** A replica starting after its peers' settle window finds everything claimed and idles as a hot
92103 spare while some peer holds two indices. See below.
93104
@@ -97,6 +108,22 @@ appearing or disappearing reshards (and therefore restarts) the entire cluster,
97108even a moment about the live count produces overlapping or missing shards. That is a materially worse failure mode
98109than one replica temporarily carrying an extra index.
99110
111+ ### Sizing a container
112+
113+ ` SHARDS_PER_REPLICA ` is the ** steady-state target, not a hard cap** , so it is not the only input to sizing. A
114+ replica holds the union of every index it claims, and it claims extras precisely when peers are missing — which is
115+ also when the surviving replicas are carrying the most load:
116+
117+ | Cluster state | Shards on the heaviest replica |
118+ | ------------------------------ | ------------------------------ |
119+ | Fully provisioned | ` SHARDS_PER_REPLICA ` |
120+ | One peer missing | ` 2 × SHARDS_PER_REPLICA ` |
121+ | Worst case (only one survivor) | the entire ` shardCount ` |
122+
123+ Size for at least ** twice** ` SHARDS_PER_REPLICA ` if a single replica loss should be absorbed without degrading,
124+ and treat ` shardsOwned ` in the boot log (and the ` covering for missing replicas ` message) as the signal that a
125+ replica is running above its target.
126+
100127### Changing shards means restarting
101128
102129` @discordjs/ws ` cannot add shards to a live ` WebSocketManager ` — ` updateShardCount ` tears everything down and
0 commit comments