Shut down idle workers early to free memory when no tests are left #1314
Description
When running with many workers (e.g., -n 24), some workers finish their share of tests well before others (for example, when you have a few extremely long running tests, and a lot of short-lived tests). Those idle workers sit around, not releasing the memory they used for the previous tests, for the rest of the run.
In our case we run a large simulation test suite with --maxprocesses=24 and each worker process holds onto a few hundred MB. When half the workers are done but the other half still have slow integration tests to grind through, we're wasting several GB of RAM on processes that will never run another test.
Proposed behavior
When a worker has no more tests to run and the scheduler confirms it won't receive any more work, shut down that worker's process instead of keeping it alive until the entire session ends.
This could be default behavior (seems like there's no reason to keep an idle worker alive), or opt-in via a flag like --early-shutdown.
Why this matters
It doesn't matter on a local machine (at least not much), But in CI, that's several GB of wasted memory on a shared machine, which could've been contributed to other teams running their tests.
It also plays well with --dist worksteal: once stealing is exhausted and a worker has nothing left. (Does it even make sense with any dist other than worksteal? Maybe load with maxschedchunk?)
I'm willing to contribute this feature, but first I'd appreciate a confirmation that this is possible, and maybe some guidance on what part of the codebase should this be injected :)