WD: Monitor crashed, this should NEVER happen! Please report this, full details in logs!

[Isratosh]

Describe the bug

WD: Monitor crashed, this should NEVER happen! Please report this, full details in logs! Recovering. Error: The maximum number of retries (6) was exceeded while executing database operations with 'MySqlRetryingExecutionStrategy'. See the inner exception for the most recent failure.

Microsoft.EntityFrameworkCore.Storage.RetryLimitExceededException: The maximum number of retries (6) was exceeded while executing database operations with 'MySqlRetryingExecutionStrategy'. See the inner exception for the most recent failure.
 ---> MySqlConnector.MySqlException (0x80004005): Connect Timeout expired.
   at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1109
   at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 567
   at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(Boolean errorsExpected, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(Boolean errorsExpected, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenAsync(CancellationToken cancellationToken, Boolean errorsExpected)
   at Pomelo.EntityFrameworkCore.MySql.Storage.Internal.MySqlRelationalConnection.OpenAsync(CancellationToken cancellationToken, Boolean errorsExpected)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQueryAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.<>c__DisplayClass30_0`2.<<ExecuteAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4 operation, Func`4 verifySucceeded, TState state, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4 operation, Func`4 verifySucceeded, TState state, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken)
   at Tgstation.Server.Host.Components.Session.SessionPersistor.ClearImpl(IDatabaseContext databaseContext, Boolean instant, CancellationToken cancellationToken) in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Components/Session/SessionPersistor.cs:line 267
   at Tgstation.Server.Host.Database.DatabaseContextFactory.UseContext(Func`2 operation) in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Database/DatabaseContextFactory.cs:line 34
   at Tgstation.Server.Host.Database.DatabaseContextFactory.UseContext(Func`2 operation) in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Database/DatabaseContextFactory.cs:line 34
   at Tgstation.Server.Host.Components.Watchdog.WatchdogBase.DisposeAndNullControllers(CancellationToken cancellationToken) in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Components/Watchdog/WatchdogBase.cs:line 744
   at Tgstation.Server.Host.Components.Watchdog.WatchdogBase.MonitorRestart(CancellationToken cancellationToken) in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Components/Watchdog/WatchdogBase.cs:line 837
   at Tgstation.Server.Host.Components.Watchdog.WatchdogBase.MonitorLifetimes(CancellationToken cancellationToken) in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Components/Watchdog/WatchdogBase.cs:line 1079
   at Tgstation.Server.Host.Components.Watchdog.WatchdogBase.StopMonitor() in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Components/Watchdog/WatchdogBase.cs:line 678
   at Tgstation.Server.Host.Components.Watchdog.WatchdogBase.TerminateNoLock(Boolean graceful, Boolean announce, CancellationToken cancellationToken) in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Components/Watchdog/WatchdogBase.cs:line 1141
   at Tgstation.Server.Host.Components.Watchdog.WatchdogBase.Restart(Boolean graceful, CancellationToken cancellationToken) in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Components/Watchdog/WatchdogBase.cs:line 409
   at Tgstation.Server.Host.Jobs.JobService.RunJob(Job job, JobEntrypoint operation, CancellationToken cancellationToken) in /home/runner/work/tgstation-server/tgstation-server/src/Tgstation.Server.Host/Jobs/JobService.cs:line 485

https://discord.com/channels/326822144233439242/386020144037756950/1486604214195982397
2026-03-26 05:54 AM tgstation terry
the us server hosting the database went down