Skip to content

Commit 64ce560

Browse files
Josef-Hauptclaude
andcommitted
Run non-TF fork tests before TF fork tests so the TF wedge can't mask them
The pt/onnx fork tests landed in the same serial -n 0 fork lane as the TF fork tests. Because a fork-after-TensorFlow wedge kills the whole -n 0 process (that is why the lane is -n 0), and perch_v2 sorts before v3_0, the intermittent TF wedge killed the lane before the pt/onnx fork tests ran -- so the very tests meant to prove fork works without TensorFlow never executed (observed in run 30630301430: fork[pt]/fork[onnx] never ran; forkserver/spawn passed in the general lane). Give the deadlock-free non-TF fork tests a "fork_nontf" marker and run them first in their own process, ahead of the TF fork tests. A wedge in the TF lane can no longer stop them from recording a result. They keep the "fork" marker too, so the parallel general lane still excludes them (forking after TF is loaded there would deadlock). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 1ca09a1 commit 64ce560

2 files changed

Lines changed: 18 additions & 5 deletions

File tree

pyproject.toml

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -208,6 +208,7 @@ markers = [
208208
"repro: tests requiring exact package versions to reproduce results",
209209
"no_tf: tests for the TensorFlow-free surface, run when TF is absent (e.g. Python 3.14)",
210210
"fork: tests that force the fork start method; run in a dedicated serial (-n 1) lane in a fresh process so a fork-after-TensorFlow deadlock cannot wedge the parallel general-phase workers (the library itself defaults to spawn, see birdnet.core.start_method)",
211+
"fork_nontf: fork tests for non-TensorFlow backends (pt/onnx), which are deadlock-free; run in their own serial invocation BEFORE the TF fork tests so the TF tests' intermittent fork-after-TensorFlow wedge cannot stop them from proving fork works without TensorFlow (also carry `fork` so the general lane still excludes them)",
211212
]
212213

213214
[tool.ruff]
@@ -285,7 +286,12 @@ commands =
285286
# with the worker, and the orphaned child keeps the worker's execnet socket open
286287
# so the controller sits blind until the watchdog's idle deadline. In-process,
287288
# the timeout report lands in the log and the run ends at the test timeout.
288-
pytest -m "not repro and not load_model and fork and not litert and not gpu" -n 0
289+
# The non-TF (pt/onnx) fork tests run first in their own invocation: they are
290+
# deadlock-free, and a fresh process means the TF fork tests' intermittent wedge
291+
# (which kills the rest of its own -n 0 run) cannot stop them from recording a
292+
# result. See the `fork_nontf` marker.
293+
pytest -m "not repro and not load_model and fork and fork_nontf and not litert and not gpu" -n 0
294+
pytest -m "not repro and not load_model and fork and not fork_nontf and not litert and not gpu" -n 0
289295
pytest -m "not repro and not load_model and fork and litert and not gpu" -n 0
290296
# build and check package
291297
# note `python -m build` uses the isolated build folder which results in import error:
@@ -302,8 +308,11 @@ commands =
302308
pytest -m "not load_model and (not litert and not gpu and not fork)" -n auto
303309
# run litert tests including repro tests
304310
pytest -m "not load_model and litert and not fork" -n auto
305-
# fork tests isolated + serial (see the `fork` marker and conftest.py)
306-
pytest -m "not load_model and fork and not litert and not gpu" -n 0
311+
# fork tests isolated + serial (see the `fork` marker and conftest.py).
312+
# Non-TF (pt/onnx) fork tests run first in their own process so the TF fork
313+
# tests' intermittent wedge cannot stop them recording a result (see fork_nontf).
314+
pytest -m "not load_model and fork and fork_nontf and not litert and not gpu" -n 0
315+
pytest -m "not load_model and fork and not fork_nontf and not litert and not gpu" -n 0
307316
pytest -m "not load_model and fork and litert and not gpu" -n 0
308317
# gpu tests are not supported to be run with repro environment
309318
pyproject-build -o dist/
@@ -328,8 +337,11 @@ commands =
328337
pytest -m "not repro and not load_model and (not litert and not gpu and not fork)" -n auto --cov=src/birdnet --cov-append --cov-report=
329338
pytest -m "not repro and not load_model and litert and not fork" -n auto --cov=src/birdnet --cov-append --cov-report=
330339
pytest -m "not repro and not load_model and gpu" -n 1 --cov=src/birdnet --cov-append --cov-report=
331-
# fork tests isolated + serial (see the `fork` marker and conftest.py)
332-
pytest -m "not repro and not load_model and fork and not litert and not gpu" -n 0 --cov=src/birdnet --cov-append --cov-report=
340+
# fork tests isolated + serial (see the `fork` marker and conftest.py).
341+
# Non-TF (pt/onnx) fork tests run first in their own process so the TF fork
342+
# tests' intermittent wedge cannot stop them recording a result (see fork_nontf).
343+
pytest -m "not repro and not load_model and fork and fork_nontf and not litert and not gpu" -n 0 --cov=src/birdnet --cov-append --cov-report=
344+
pytest -m "not repro and not load_model and fork and not fork_nontf and not litert and not gpu" -n 0 --cov=src/birdnet --cov-append --cov-report=
333345
pytest -m "not repro and not load_model and fork and litert and not gpu" -n 0 --cov=src/birdnet --cov-append --cov-report=
334346
coverage report
335347
"""

src/birdnet_tests/acoustic_models/v3_0/model_py/test_predict/test_start_method_v3_0.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,7 @@ def _run_two_parallel_sessions(backend: _Backend) -> None:
6969
# it moves from TensorFlow to PyTorch, so these lanes must stay green; they are the
7070
# coverage that proves fork works for everything except TensorFlow.
7171
@pytest.mark.fork
72+
@pytest.mark.fork_nontf
7273
@pytest.mark.parametrize("backend", ["pt", "onnx"])
7374
def test_twice_two_sessions_parallel_processes_fork(backend: _Backend) -> None:
7475
# macOS is skipped to match the perch/v2.4 fork tests: forking the

0 commit comments

Comments
 (0)