Commit b8766d8
committed
Use the "spawn" start method instead of forkserver
Python 3.14 defaults to the "forkserver" multiprocessing start method,
instead of the previous "fork" default. This new default causes all
sorts of problems for us.
The hardest to fix one, which made me throw in the towel on trying to
get things working with "forkserver", is that pytest's `pytester`
fixture defaults to running the new pytest session in the current
process, rather than from a subprocess, and we're using `capfd` to
capture the stderr. With the "fork" method and the "spawn" method, the
new process inherits whatever the parent process's stderr (fd 2) is
pointed to at the time when the new process is started, but with
"forkserver" the server gets created once and then reused, and all
future children have their stderr pointed wherever the first child
process's stderr was pointed to.
There's other issues too. We leak semaphores because the monitoring
process winds up creating an unnecessary `Event` and `Queue` that it
ignores in favor of the ones passed to its entry point. That could be
fixed by creating those lazily instead of at import time, but that leads
to pickling failures because the `multiprocessing.connection` module
gets created lazily, and `pytester` unloads modules that were loaded by
the tests, which leads to two different copies of that module getting
loaded, with the old module still referenced by some things.
We could work around all of this in our test suite, but using our plugin
along with the `pytester` fixture is a reasonable use case that end
users might want, if they're trying to use `pytest-pystack` to test
their own pytest plugin.
The most reasonable solution is for us to detect when "forkserver" would
be used, and to use "spawn" instead in those cases.1 parent 6306d63 commit b8766d8
3 files changed
Lines changed: 13 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | 2 | | |
4 | 3 | | |
5 | | - | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
6 | 7 | | |
7 | 8 | | |
8 | 9 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
2 | 1 | | |
3 | 2 | | |
4 | 3 | | |
| |||
7 | 6 | | |
8 | 7 | | |
9 | 8 | | |
| 9 | + | |
10 | 10 | | |
11 | | - | |
| 11 | + | |
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
| 17 | + | |
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
0 commit comments