Skip the bfloat16 GPT tests on Windows

Six tests intermittently kill their xdist worker on windows-latest:

  Windows fatal exception: code 0xc000001d
  [gw1] node down: Not properly terminated

Always the same six, on about half the runs, on released and nightly
torch alike. The branch has also passed with the same torch on other
runs, so this follows the machine the job lands on rather than anything
in the tree.

The fault is in eager torch, not in our code. For test_litgpt the
innermost frames are torch's Linear.forward, reached from the plain
reference call before the jitted function is ever invoked; the
interpreter variants fault in the same place through the opaque-call
path. What separates these six from the tests that pass is bfloat16:
test_litgpt_variants builds the same gpt-neox-like model in float32 and
passes in the same runs that crash test_litgpt.

The previous attempt set ATEN_CPU_CAPABILITY=default on Windows, on the
theory that the wheels carry AVX-512 instructions inside kernels
compiled for the AVX2 target. A diagnostic step confirmed the capability
applied and all six still crashed, so the fault is not in ATen's
ISA-dispatched kernels. That does not clear the wider codegen issue in
pytorch/pytorch#145702: bfloat16 matmuls go to oneDNN or MKL, which pick
their own instruction set and ignore that variable. Those paths are
untested, so the workflow change is reverted rather than extended.

Skip the six on Windows until this is understood. Nothing in process can
retry them: the worker dies outright and pytest only sees a dead
channel, so rerunfailures and xfail both have nothing to act on.

Author: bhimrazy

.github/workflows/ci-testing.yml

@@ -45,16 +45,6 @@ jobs:
     # Timeout: https://stackoverflow.com/a/59076067/4521646
     timeout-minutes: 35
 
-    env:
-      # torch's Windows CPU wheels carry AVX-512 instructions inside kernels compiled for the
-      # AVX2 target, so on a runner whose CPU stops at AVX2 they fault with
-      # STATUS_ILLEGAL_INSTRUCTION (0xc000001d) and take the xdist worker down with them. The
-      # reported fault sits in the bfloat16 vector helpers, which is why the nanogpt and litgpt
-      # tests are the ones that crash: they are the only ones here running a model in bfloat16.
-      # Fall back to the unvectorized kernels on Windows until this is fixed upstream.
-      # Ref: https://github.com/pytorch/pytorch/issues/145702
-      ATEN_CPU_CAPABILITY: ${{ startsWith(matrix.os, 'windows') && 'default' || '' }}
-
     steps:
       - uses: actions/checkout@v5
       - name: Set up Python ${{ matrix.python-version }}
@@ -111,12 +101,6 @@ jobs:
             --extra-index-url=${TORCH_URL}
           pip list
 
-      - name: Show CPU dispatch
-        # So a green Windows run can be told apart from a run that happened to land on a CPU
-        # the bad kernels do not fault on. Expect DEFAULT here, not AVX2.
-        if: runner.os == 'Windows'
-        run: python -c "import torch; print('cpu capability:', torch.backends.cpu.get_cpu_capability())"
-
       - name: Testing Local
         if: matrix.suite == 'core'
         run: |

thunder/tests/framework.py

@@ -58,6 +58,11 @@ class NOTHING:
 )
 IS_WINDOWS = platform.system() == "Windows"
 
+# NOTE Running these models in bfloat16 on CPU intermittently kills the process on Windows with an
+#   illegal instruction, inside eager torch rather than anything of ours. The fp32 variants of the
+#   same models still run there, and these still run on Linux and macOS.
+WINDOWS_GPT_CRASH_REASON = "bfloat16 CPU model intermittently crashes the process on Windows (0xc000001d)"
+
 
 def _bitsandbytes_available():
     if not package_available("bitsandbytes"):

thunder/tests/test_interpreter.py

@@ -11,7 +11,7 @@
 import pytest
 import torch
 from torch.testing import assert_close
-from thunder.tests.framework import IS_WINDOWS
+from thunder.tests.framework import IS_WINDOWS, WINDOWS_GPT_CRASH_REASON
 
 import thunder
 from thunder.core.interpreter import (
@@ -3413,6 +3413,7 @@ def test_nanogpt_block(jit):
     assert_close(result, fn(*args, **kwargs))
 
 
+@pytest.mark.skipif(IS_WINDOWS, reason=WINDOWS_GPT_CRASH_REASON)
 def test_nanogpt(jit):
     from thunder.benchmarks import NanoGPTBenchmark, NanoGPTConfig, _nanogpt_configs
 
@@ -3428,6 +3429,7 @@ def test_nanogpt(jit):
     assert_close(result, fn(*args, **kwargs))
 
 
+@pytest.mark.skipif(IS_WINDOWS, reason=WINDOWS_GPT_CRASH_REASON)
 def test_litgpt(jit):
     from thunder.benchmarks import LitGPTBenchmark
     from thunder.tests.litgpt_model import Config

thunder/tests/test_jit_general.py

@@ -14,7 +14,7 @@
 
 import thunder
 
-from thunder.tests.framework import requiresCUDA, IS_WINDOWS
+from thunder.tests.framework import requiresCUDA, IS_WINDOWS, WINDOWS_GPT_CRASH_REASON
 from thunder.core.options import CACHE_OPTIONS
 import thunder.core.prims as prims
 from thunder import pytorch_executor, nvfuser_executor
@@ -585,6 +585,7 @@ def h(d, c):
     assert args_names == ("a", "b", "c", "d")
 
 
+@pytest.mark.skipif(IS_WINDOWS, reason=WINDOWS_GPT_CRASH_REASON)
 def test_litgpt():
     from thunder.benchmarks import LitGPTBenchmark
     from thunder.tests.litgpt_model import Config
@@ -665,6 +666,7 @@ def test_nanogpt_mlp():
     assert_close(result, module(*args, **kwargs))
 
 
+@pytest.mark.skipif(IS_WINDOWS, reason=WINDOWS_GPT_CRASH_REASON)
 def test_nanogpt():
     from thunder.benchmarks import NanoGPTBenchmark, NanoGPTConfig, _nanogpt_configs