Remove legacy cluster auto-downing

BREAKING_CHANGES_V1.6.md

@@ -40,6 +40,7 @@ of `Behavior`, `Wire`, `API` (combine with `+`).
 
 | Status | PR / Branch | Component | Type | Change | Migration |
 |--------|-------------|-----------|------|--------|-----------|
+| Planned | `docs/8439-auto-downing-investigation` | `Akka.Cluster` | Behavior + API | Removed the unsafe legacy auto-down implementation, including the public `Akka.Cluster.AutoDowning` type and `ClusterSettings.AutoDownUnreachableAfter`. The `akka.cluster.auto-down-unreachable-after` setting is ignored; active values emit a warning, while `off`, `false`, and `no` remain quiet. Explicitly selecting the removed provider fails with a targeted `ConfigurationException`. | Use the default Split Brain Resolver or configure another production `IDowningProvider`; there is no duration-for-duration mapping from auto-down to SBR. Tests that deliberately need deterministic unsafe downing can reference `Akka.Cluster.TestKit` and compose `AutoDowning.GetConfig(delay)`. |
 | Planned | [#8324](https://github.com/akkadotnet/akka.net/pull/8324) | `Akka.Routing` | Behavior | `ConsistentHash<T>` no longer retains the `SortedDictionary` passed to its public constructor — the ring is snapshotted into internal sorted arrays. Mutating that dictionary after construction no longer affects the instance (previously the aliasing was inconsistent: `IsEmpty` and `operator +`/`-` read it live, while `NodeFor` froze it after the first lookup). A `null` dictionary now throws `ArgumentNullException` from the constructor instead of surfacing later as a `NullReferenceException`. No public API removed; the ring built by `ConsistentHash.Create` is byte-identical. (#8293) | None for normal use — `ConsistentHash.Create` already builds the dictionary fully before constructing, so routers/receptionists are unaffected. If you call the `ConsistentHash(SortedDictionary, int)` constructor directly, populate the dictionary before passing it and don't rely on post-construction mutation being visible. |
 | Planned | `fix/gossip-removal-tombstones` | `Akka.Cluster` | Wire + Behavior | Cluster gossip now carries a removal tombstone for every member the leader removes (new `repeated Tombstone tombstones = 7` on the `Gossip` proto message, previously the one vacant field number). A member the leader removed can no longer be put back into the gossip by a peer that has not caught up, which previously blocked convergence permanently and needed a full cluster restart to clear. Tombstones expire after the new `akka.cluster.prune-gossip-tombstones-after` setting (default `24h`), pruned by the leader on a converged tick and reclaimed for good: gossip reception keeps the tombstones of whichever gossip wins the causal comparison, so a peer that has not pruned yet cannot hand a pruned tombstone back. Tombstones are only unioned across the two gossips when neither descends from the other -- equal clocks, or concurrent clocks, where each side may hold a removal the other has not heard about. The field is additive and proto3 ignores unknown fields in both directions, so rolling upgrades are safe; an older node parses the gossip fine but drops the tombstones when it re-emits it, and if that stripped gossip is causally newer than an upgraded node's, the upgraded node adopts it and loses those tombstones too -- so the protection only holds between upgraded nodes and the fix is not fully in effect until every node is upgraded. | No action required. Raise `akka.cluster.prune-gossip-tombstones-after` if the cluster must survive partitions longer than 24 hours, or lower it if a cluster with heavy join/leave churn needs the gossip message kept small -- each tombstone adds a full address plus a timestamp to every gossip message until it expires. |
 | Planned | `feature/default-bounded-shard-rebalancing` | `Akka.Cluster.Sharding` | Behavior | The default `rebalance-absolute-limit` is now `20`, selecting the bounded shard allocation strategy instead of the legacy threshold-based strategy. | To retain the legacy strategy temporarily, explicitly set `akka.cluster.sharding.least-shard-allocation-strategy.rebalance-absolute-limit = 0`. Review `rebalance-threshold` and `max-simultaneous-rebalance`, which do not apply while the bounded strategy is active. |

docs/articles/clustering/cluster-overview.md

@@ -223,7 +223,7 @@ Each role within the cluster also has a leader, just for that role. Its primary
 
 ### Reachability
 
-Nodes send each other [heartbeats](<https://en.wikipedia.org/wiki/Heartbeat_(computing)>) on an ongoing basis. If a node misses enough heartbeats, this will trigger `unreachable` gossip messages from its peers. The leader will wait for the node to either become reachable again, restart or get downed. Until that happens, the cluster is not in a consistent state and the leader indicates that it is unable to perform its duties. If the gossip from a quorum of cluster nodes agree that the node is unreachable ("convergence"), the leader will mark it as down and begin removing the node from the cluster. You can control how long the cluster waits for unreachable nodes through the auto-down-unreachable-after setting.
+Nodes send each other [heartbeats](<https://en.wikipedia.org/wiki/Heartbeat_(computing)>) on an ongoing basis. If a node misses enough heartbeats, this will trigger `unreachable` gossip messages from its peers. The leader will wait for the node to either become reachable again, restart, or get downed. Until that happens, the cluster is not in a consistent state and the leader indicates that it is unable to perform its duties. The configured `IDowningProvider` decides which unreachable members to down and when; Akka.NET uses the [keep-majority Split Brain Resolver](xref:split-brain-resolver) by default. The legacy `auto-down-unreachable-after` setting was removed in Akka.NET v1.6 and is ignored.
 
 When marked as unreachable, the node can restart and join the cluster again, however the association will only be formed if that node is identified as the same node that became unreachable. If you use dynamic addressing (port 0), starting a node again might result in a different port being assigned upon restart. The result of that is that the cluster remains in an inconsistent state, waiting to the unreachable node to either become reachable or get downed.
 

docs/articles/clustering/split-brain-resolver.md

@@ -36,7 +36,10 @@ akka.cluster {
 }
 ```
 
-Keep in mind that split brain resolver will NOT work when `akka.cluster.auto-down-unreachable-after` is used.
+The legacy `akka.cluster.auto-down-unreachable-after` mechanism was removed in Akka.NET v1.6.
+Remove that setting and configure the Split Brain Resolver instead. Do not copy the former
+auto-down delay into `stable-after`: auto-down and SBR use different decision models, so choose
+`stable-after` based on your failure detector, expected transient outages, and operational needs.
 
 ## Split Brain Resolution Strategies
 

docs/community/whats-new/akkadotnet-v1.5-upgrade-advisories.md

@@ -139,6 +139,12 @@ The `auto-down-unreachable-after` feature has been deprecated as of Akka.NET v1.
 The `keep-majority` split brain resolver will be used instead. See https://getakka.net/articles/cluster/split-brain-resolver.html for more details.
 ```
 
+Starting with Akka.NET v1.6, the setting has been removed. An active
+`akka.cluster.auto-down-unreachable-after` value is ignored and produces a migration warning;
+`off`, `false`, and `no` remain quiet. Remove the setting and use the default Split Brain Resolver
+or configure another production `IDowningProvider`. An auto-down duration does not map directly to
+an SBR `stable-after` value because the two mechanisms make different partition decisions.
+
 #### Disabling the Default Downing Provider
 
 To disable the default Akka.Cluster downing provider, simply configure the following in your HOCON:

src/contrib/cluster/Akka.Cluster.Sharding.Tests.MultiNode/ClusterShardingRememberEntitiesNewExtractorSpec.cs

@@ -9,6 +9,7 @@
 using System.Linq;
 using System.Threading.Tasks;
 using Akka.Actor;
+using Akka.Cluster.TestKit;
 using Akka.Configuration;
 using Akka.Event;
 using Akka.MultiNode.TestAdapter;
@@ -25,7 +26,7 @@ public class ClusterShardingRememberEntitiesNewExtractorSpecConfig : MultiNodeCl
     public RoleName Third { get; }
 
     public ClusterShardingRememberEntitiesNewExtractorSpecConfig(StateStoreMode mode)
-        : base(mode: mode, loglevel: "DEBUG")
+        : base(mode: mode, loglevel: "DEBUG", configurationOverride: AutoDowning.GetConfig(TimeSpan.Zero))
     {
         First = Role("first");
         Second = Role("second");
@@ -337,4 +338,4 @@ await RunOnAsync(async () =>
             await EnterBarrierAsync("done");
         });
     }
-}
+}

src/contrib/cluster/Akka.Cluster.Sharding.Tests.MultiNode/ClusterShardingRememberEntitiesSpec.cs

@@ -9,6 +9,7 @@
 using System.Threading;
 using System.Threading.Tasks;
 using Akka.Actor;
+using Akka.Cluster.TestKit;
 using Akka.Configuration;
 using Akka.MultiNode.TestAdapter;
 using Akka.Remote.TestKit;
@@ -32,7 +33,7 @@ public ClusterShardingRememberEntitiesSpecConfig(
             loglevel: "DEBUG", additionalConfig: @"
               akka.testconductor.barrier-timeout = 60 s
               akka.test.single-expect-default = 60 s
-            ")
+            ", configurationOverride: AutoDowning.GetConfig(TimeSpan.Zero))
     {
         First = Role("first");
         Second = Role("second");
@@ -286,4 +287,4 @@ await RunOnAsync(async () =>
             await EnterBarrierAsync("after-3");
         });
     }
-}
+}

src/contrib/cluster/Akka.Cluster.Sharding.Tests.MultiNode/ClusterShardingSpec.cs

@@ -11,6 +11,7 @@
 using System.Threading.Tasks;
 using Akka.Actor;
 using Akka.Cluster.Sharding.Internal;
+using Akka.Cluster.TestKit;
 using Akka.Cluster.Tools.Singleton;
 using Akka.Configuration;
 using Akka.DistributedData;
@@ -59,7 +60,6 @@ public ClusterShardingSpecConfig(
         CommonConfig = ConfigurationFactory.ParseString($@"
                 akka.cluster.sharding.verbose-debug-logging = on
                 #akka.loggers = [""akka.testkit.SilenceAllTestEventListener""]
-                akka.cluster.auto-down-unreachable-after = 0s
                 akka.cluster.roles = [""backend""]
                 akka.cluster.distributed-data.gossip-interval = 1s
                 akka.persistence.journal.sqlite-shared.timeout = 10s #the original default, base test uses 5s
@@ -84,7 +84,10 @@ public ClusterShardingSpecConfig(
                     }}
                 }}
                 akka.testconductor.barrier-timeout = 70s
-              ").WithFallback(PersistenceConfig()).WithFallback(Common);
+              ")
+            .WithFallback(AutoDowning.GetConfig(TimeSpan.Zero))
+            .WithFallback(PersistenceConfig())
+            .WithFallback(Common);
 
         NodeConfig(new[] { Sixth }, new[] { ConfigurationFactory.ParseString(@"akka.cluster.roles = [""frontend""]") });
     }
@@ -1285,4 +1288,4 @@ await AwaitAssertAsync(async () =>
     }
 
     #endregion
-}
+}

src/contrib/cluster/Akka.Cluster.Sharding.Tests.MultiNode/MultiNodeClusterShardingConfig.cs

@@ -40,12 +40,14 @@ class = ""Akka.Cluster.Sharding.Tests.MemorySnapshotStoreShared, Akka.Cluster.Sh
         /// <param name="rememberEntities">rememberEntities defaults to off</param>
         /// <param name="additionalConfig">additionalConfig additional _config</param>
         /// <param name="loglevel">loglevel defaults to INFO</param>
+        /// <param name="configurationOverride">optional configuration with higher precedence than the shared configuration</param>
         protected MultiNodeClusterShardingConfig(
             StateStoreMode mode = StateStoreMode.DData,
             bool rememberEntities = false,
             RememberEntitiesStore rememberEntitiesStore = RememberEntitiesStore.DData,
             string additionalConfig = "",
-            string loglevel = "INFO")
+            string loglevel = "INFO",
+            Config configurationOverride = null)
         {
             Mode = mode;
             RememberEntities = rememberEntities;
@@ -58,7 +60,6 @@ protected MultiNodeClusterShardingConfig(
             Common =
                 ConfigurationFactory.ParseString($@"
                     akka.actor.provider = ""cluster""
-                    akka.cluster.auto-down-unreachable-after = 0s
                     akka.cluster.sharding.state-store-mode = ""{mode}""
                     akka.cluster.sharding.remember-entities = {rememberEntities.ToString().ToLowerInvariant()}
                     akka.cluster.sharding.remember-entities-store = ""{rememberEntitiesStore}""
@@ -84,7 +85,10 @@ protected MultiNodeClusterShardingConfig(
                     .WithFallback(Tools.Singleton.ClusterSingleton.DefaultConfig())
                     .WithFallback(MultiNodeClusterSpec.ClusterConfig());
 
-            CommonConfig = (ConfigurationFactory.ParseString(additionalConfig).WithFallback(persistenceConfig).WithFallback(Common));
+            CommonConfig = (configurationOverride ?? ConfigurationFactory.Empty)
+                .WithFallback(ConfigurationFactory.ParseString(additionalConfig))
+                .WithFallback(persistenceConfig)
+                .WithFallback(Common);
         }
 
         public StateStoreMode Mode { get; }

src/contrib/cluster/Akka.Cluster.Tools.Tests.MultiNode/ClusterClient/ClusterClientSpec.cs

@@ -48,7 +48,6 @@ public ClusterClientSpecConfig()
                 akka.testconductor.query-timeout = 1m # we were having timeouts shutting down nodes with 5s default
                 akka.actor.provider = cluster
                 akka.remote.log-remote-lifecycle-events = off
-                akka.cluster.auto-down-unreachable-after = 0s
                 akka.cluster.failure-detector.acceptable-heartbeat-pause = 6s # de-flake: cluster-level FD (distinct from client FD below); absorb CI stalls during formation
                 akka.cluster.client.heartbeat-interval = 1s
                 akka.cluster.client.acceptable-heartbeat-pause = 3s
@@ -60,6 +59,7 @@ public ClusterClientSpecConfig()
                 akka.cluster.client.receptionist.failure-detection-interval = 1s
                 akka.test.filter-leeway = 10s
             ")
+            .WithFallback(AutoDowning.GetConfig(TimeSpan.Zero))
             .WithFallback(ClusterClientReceptionist.DefaultConfig())
             .WithFallback(DistributedPubSub.DefaultConfig());
 
@@ -698,4 +698,4 @@ await RunOnAsync(async () =>
             }, _remainingServerRoleNames.ToArray());
         });
     }
-}
+}

src/contrib/cluster/Akka.Cluster.Tools.Tests.MultiNode/PublishSubscribe/DistributedPubSubMediatorSpec.cs

@@ -42,11 +42,12 @@ public DistributedPubSubMediatorSpecConfig()
                 akka.actor.provider = ""Akka.Cluster.ClusterActorRefProvider, Akka.Cluster""
                 akka.actor.serialize-messages = off
                 akka.remote.log-remote-lifecycle-events = off
-                akka.cluster.auto-down-unreachable-after = 0s
-                akka.cluster.failure-detector.acceptable-heartbeat-pause = 6s # de-flake: absorb CI stalls during formation (default 3s); keep 0s auto-down (spec crashes nodes)
+                akka.cluster.failure-detector.acceptable-heartbeat-pause = 6s # de-flake: absorb CI stalls during formation (default 3s); keep zero-delay TestKit downing (spec crashes nodes)
                 akka.cluster.pub-sub.max-delta-elements = 500
                 akka.testconductor.query-timeout = 1m # we were having timeouts shutting down nodes with 5s default
-            ").WithFallback(DistributedPubSub.DefaultConfig());
+            ")
+            .WithFallback(AutoDowning.GetConfig(TimeSpan.Zero))
+            .WithFallback(DistributedPubSub.DefaultConfig());
     }
 }
 
@@ -784,4 +785,4 @@ await RunOnAsync(async () =>
             await EnterBarrierAsync("after-15");
         });
     }
-}
+}

src/contrib/cluster/Akka.Cluster.Tools.Tests.MultiNode/PublishSubscribe/DistributedPubSubPublishWithAckSpec.cs

@@ -35,7 +35,6 @@ public DistributedPubSubPublishWithAckSpecSpecConfig()
             akka.actor.serialize-messages = off
             akka.remote.log-remote-lifecycle-events = off
             akka.cluster.failure-detector.acceptable-heartbeat-pause = 6s # de-flake: absorb CI heartbeat stalls during formation (default 3s)
-            akka.cluster.auto-down-unreachable-after = 5s
             akka.cluster.pub-sub.max-delta-elements = 500
             akka.cluster.pub-sub.buffered-messages.max-per-topic = 2
             akka.cluster.pub-sub.buffered-messages.timeout-check-interval = 200ms

src/contrib/cluster/Akka.Cluster.Tools.Tests.MultiNode/PublishSubscribe/DistributedPubSubRestartSpec.cs

@@ -37,7 +37,6 @@ public DistributedPubSubRestartSpecConfig()
                 akka.actor.provider = ""Akka.Cluster.ClusterActorRefProvider, Akka.Cluster""
                 akka.cluster.pub-sub.gossip-interval = 500ms
                 akka.remote.log-remote-lifecycle-events = off
-                akka.cluster.auto-down-unreachable-after = off
 
                 # THE fix for this spec's flake. Bound the transport failure detector so a peer that
                 # dies without a clean Disassociate PDU gets reaped fast. Under the test transport
@@ -346,4 +345,4 @@ await AwaitAssertAsync(async () =>
             (await probe.ExpectMsgAsync<int>(1.Seconds())).Should().Be(expected);
         }, 10.Seconds(), 500.Milliseconds());
     }
-}
+}

src/contrib/cluster/Akka.Cluster.Tools.Tests.MultiNode/Singleton/ClusterSingletonManagerChaosSpec.cs

@@ -45,9 +45,9 @@ public ClusterSingletonManagerChaosConfig()
                 akka.loglevel = DEBUG
                 akka.actor.provider = ""Akka.Cluster.ClusterActorRefProvider, Akka.Cluster""
                 akka.remote.log-remote-lifecycle-events = off
-                akka.cluster.auto-down-unreachable-after = 0s
-                akka.cluster.failure-detector.acceptable-heartbeat-pause = 6s # de-flake: absorb CI stalls during formation (default 3s); keep 0s auto-down (spec crashes nodes)
+                akka.cluster.failure-detector.acceptable-heartbeat-pause = 6s # de-flake: absorb CI stalls during formation (default 3s); keep zero-delay TestKit downing (spec crashes nodes)
             ")
+            .WithFallback(AutoDowning.GetConfig(TimeSpan.Zero))
             .WithFallback(ClusterSingleton.DefaultConfig())
             .WithFallback(ClusterSingletonProxy.DefaultConfig())
             .WithFallback(MultiNodeClusterSpec.ClusterConfig());

src/contrib/cluster/Akka.Cluster.Tools.Tests.MultiNode/Singleton/ClusterSingletonManagerDownedSpec.cs

@@ -36,7 +36,6 @@ public ClusterSingletonManagerDownedSpecConfig()
                 akka.loglevel = INFO
                 akka.actor.provider = ""Akka.Cluster.ClusterActorRefProvider, Akka.Cluster""
                 akka.remote.log-remote-lifecycle-events = off
-                akka.cluster.auto-down-unreachable-after = off
             ")
             .WithFallback(ClusterSingleton.DefaultConfig())
             .WithFallback(ClusterSingletonProxy.DefaultConfig())
@@ -190,4 +189,4 @@ await RunOnAsync(async () =>
 
         await EnterBarrierAsync("stopped");
     }
-}
+}

src/contrib/cluster/Akka.Cluster.Tools.Tests.MultiNode/Singleton/ClusterSingletonManagerLeaseSpec.cs

@@ -48,11 +48,8 @@ public ClusterSingletonManagerLeaseSpecConfig()
                 # single false unreachable instantly and permanently downed a joining node with zero
                 # grace, and the 'wait for N members Up' assertions in form_a_cluster would fail.
                 # This spec removes nodes only via explicit Cluster.Down(...) (see the oldest-node
-                # phase), so it never relied on auto-down to reap a genuinely-crashed node; a small
-                # auto-down grace is therefore safe and keeps transient blips from being fatal.
+                # phase), so it does not need an automatic downing provider.
                 akka.cluster.failure-detector.acceptable-heartbeat-pause = 6s
-                akka.cluster.auto-down-unreachable-after = 5s
-                akka.cluster.testkit.auto-down-unreachable-after = 5s
                 test-lease {
                     lease-class = ""Akka.TestKit.TestLeaseActorClient, Akka.Tests.Shared.Internals.Xunit3""
                     heartbeat-interval = 1s

src/contrib/cluster/Akka.Cluster.Tools.Tests.MultiNode/Singleton/ClusterSingletonManagerLeave2Spec.cs

@@ -42,7 +42,6 @@ public ClusterSingletonManagerLeave2SpecConfig()
                 akka.loglevel = INFO
                 akka.actor.provider = ""cluster""
                 akka.remote.log-remote-lifecycle-events = off
-                akka.cluster.auto-down-unreachable-after = off
             ")
                 .WithFallback(ClusterSingleton.DefaultConfig())
                 .WithFallback(ClusterSingletonProxy.DefaultConfig())

src/contrib/cluster/Akka.Cluster.Tools.Tests.MultiNode/Singleton/ClusterSingletonManagerLeaveSpec.cs

@@ -35,7 +35,6 @@ public ClusterSingletonManagerLeaveSpecConfig()
                 akka.loglevel = INFO
                 akka.actor.provider = ""Akka.Cluster.ClusterActorRefProvider, Akka.Cluster""
                 akka.remote.log-remote-lifecycle-events = off
-                akka.cluster.auto-down-unreachable-after = off
 
                 # The harness already runs gossip and leader actions at 200ms
                 # (MultiNodeClusterSpec.ClusterConfig). Left at the 1s default, the