Decide whether a timed-out replacement may leave joint consensus

[gaborbernat]

Question

Now that the membership commit is bounded (#2215, PR #2224), a ReplaceVoter that times out leaves the cluster in a state nobody has decided is correct.

Observed and asserted rather than assumed: after the bound fires, cluster_status().voters is ["east", "west"]. The commit stops inside joint consensus, with the replaced voter still present alongside the promoted one.

That is arguably the good outcome — the cluster does not strand itself on the node that went quiet, and a caller who retries can make progress. But joint consensus is normally a transient state on the way to a new configuration, and nothing here decides how long it may persist or what clears it.

What needs deciding

• Is a standing joint configuration after a failed replacement intended, or should the change be rolled back to the original voters on timeout?
• If it is intended, what clears it — a retry of the same ReplaceVoter, an operator action, or something automatic?
• Does a cluster left in joint consensus behave correctly for reads and writes indefinitely, or does it carry a cost that makes lingering there unacceptable?

Why this is a question rather than a bug report

Nothing observed is misbehaving. The bound is new, so this state is newly reachable, and the person who introduced it deliberately did not touch the design question. Answering it may well be "this is correct, document it" — but that answer should be written down rather than assumed by the next person to read the code.

If the answer is that it should roll back, that is a separate change with its own risks: a rollback that itself needs a quorum can fail the same way, and a failed rollback after a failed commit is worse than a standing joint configuration.

Acceptance criteria

• A decision is recorded, in code comments or docs, on whether a timed-out replacement may leave joint consensus standing.
• If it may, what clears it is stated.
• If it may not, the rollback path is specified along with what happens when the rollback itself cannot commit.

Boundary

The decision and its documentation. Do not change the bound introduced by #2215, and do not add a rollback without deciding the question first.