ThunderFX: Modify GraphModule in-place - #2399
Merged
shino16 merged 8 commits intoAug 28, 2025
Merged
Lit OSS [bot] / tests.yaml / Lit Job (pytorchlightning/lightning-thunder:ubuntu24.04-cuda12.6.3-cudnn-fe1.10.0-py3.10-pt_main-dev, distributed, L4_X_2)
succeeded
Aug 28, 2025 in 27m 17s
Job finished
Job 'ci-run_Lightning-AI-lightning-thunder-6555495-event-564c6b30-tests.yaml_/Lit_Job(pytorchlightning/lightning-thunder:ubuntu24.04-cuda12.6.3-cudnn-fe1.10.0-py3.10-pt_main-dev,_distributed,_L4_X_2)' finished as Completed with exit code 0.
Details
nvidia:
Thu Aug 28 13:14:46 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.163.01 Driver Version: 550.163.01 CUDA Version: 12.6 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA L4 Off | 00000000:00:04.0 Off | 0 |
| N/A 39C P8 11W / 72W | 1MiB / 23034MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA L4 Off | 00000000:00:05.0 Off | 0 |
| N/A 45C P8 12W / 72W | 1MiB / 23034MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+
Python 3.10.18
pip 25.2 from /usr/local/lib/python3.10/dist-packages/pip (python 3.10)
Package Version
--------------------- ------------------
absl-py 2.3.1
accelerate 1.10.1
aiohappyeyeballs 2.6.1
aiohttp 3.12.15
aiosignal 1.4.0
apex 0.1
asv 0.6.4
asv_runner 0.2.1
asvdb 0.4.2
async-timeout 5.0.1
attrs 25.3.0
bitsandbytes 0.46.1
blinker 1.7.0
boto3 1.40.18
botocore 1.40.18
build 1.3.0
certifi 2025.8.3
charset-normalizer 3.4.3
cmake 3.31.6
coverage 7.9.2
cryptography 41.0.7
dbus-python 1.3.2
diffusers 0.34.0
distlib 0.4.0
distro 1.9.0
docstring_parser 0.17.0
einops 0.8.1
exceptiongroup 1.3.0
execnet 2.1.1
expecttest 0.3.0
fdm 0.5.0
filelock 3.19.1
frozenlist 1.7.0
fsspec 2025.7.0
graphviz 0.21
hf-xet 1.1.8
httplib2 0.20.4
huggingface-hub 0.34.4
hypothesis 6.136.9
idna 3.10
importlib_metadata 8.7.0
importlib_resources 6.5.2
iniconfig 2.1.0
Jinja2 3.1.6
jmespath 1.0.1
json5 0.12.1
jsonargparse 4.32.1
launchpadlib 1.11.0
lazr.restfulclient 0.14.6
lazr.uri 1.0.6
lightning 2.4.0
lightning-utilities 0.15.2
lintrunner 0.12.7
litgpt 0.5.0
MarkupSafe 3.0.2
mpmath 1.3.0
multidict 6.6.4
networkx 3.4.2
ninja 1.13.0
numpy 1.26.4
nvfuser 0.2.30+git6d4256e
nvidia-cudnn-frontend 1.10.0
oauthlib 3.2.2
optree 0.17.0
packaging 24.2
pandas 2.3.2
pillow 11.3.0
pip 25.2
platformdirs 4.4.0
pluggy 1.6.0
propcache 0.3.2
psutil 7.0.0
py-cpuinfo 9.0.0
pybind11 3.0.1
pybind11-global 3.0.1
PyGObject 3.48.2
PyJWT 2.7.0
Pympler 1.1
pyparsing 3.1.1
pyproject_hooks 1.2.0
pytest 8.3.5
pytest-benchmark 5.1.0
pytest-cov 6.2.1
pytest-random-order 1.2.0
pytest-timeout 2.4.0
pytest-timestamper 0.0.10
pytest-xdist 3.7.0
python-apt 2.7.7+ubuntu5
python-dateutil 2.9.0.post0
pytorch-lightning 2.5.3
pytorch-triton 3.4.0+gitf7888497
pytz 2025.2
PyYAML 6.0.2
regex 2025.7.34
requests 2.32.5
s3transfer 0.13.1
safetensors 0.6.2
setuptools 79.0.1
six 1.16.0
sortedcontainers 2.4.0
sympy 1.14.0
tabulate 0.9.0
tokenizers 0.21.4
tomli 2.2.1
torch 2.9.0a0+git3d40642
torchmetrics 1.8.1
tqdm 4.67.1
transformers 4.52.4
typeshed_client 2.8.2
typing_extensions 4.15.0
tzdata 2025.2
urllib3 2.5.0
uv 0.8.13
virtualenv 20.34.0
wadllib 1.3.6
wheel 0.45.1
xlsxwriter 3.2.5
yarl 1.20.1
zipp 3.23.0
+ bash scripts/remove-torch-lines.sh requirements/base.txt
+ cat requirements/base.txt
looseversion ==1.3.0
lightning-utilities >0.7.0
numpy
networkx >=3.3
optree >=0.12.1
opt_einsum >=3.3.0
# todo: teporarl pin for `NameError: name '_C' is not defined`
mpmath <1.4.0
# Support for 3.12
dill >=0.3.8
+ pip install -U -r requirements/base.txt -r requirements/test.txt
Ignoring bitsandbytes: markers '"arm" in platform_machine or "aarch" in platform_machine' don't match your environment
Collecting asvdb@ git+https://github.com/rapidsai/asvdb.git (from -r requirements/test.txt (line 26))
Cloning https://github.com/rapidsai/asvdb.git to /tmp/pip-install-ag1emjqe/asvdb_0fe64bef00ce487e802800c1060c9799
Running command git clone --filter=blob:none --quiet https://github.com/rapidsai/asvdb.git /tmp/pip-install-ag1emjqe/asvdb_0fe64bef00ce487e802800c1060c9799
Resolved https://github.com/rapidsai/asvdb.git to commit fd1616a47f9b37bdce4c9c61a5bdd9310daccb72
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Collecting looseversion==1.3.0 (from -r requirements/base.txt (line 1))
Downloading looseversion-1.3.0-py2.py3-none-any.whl.metadata (4.6 kB)
Requirement already satisfied: lightning-utilities>0.7.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements/base.txt (line 2)) (0.15.2)
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from -r requirements/base.txt (line 3)) (1.26.4)
Collecting numpy (from -r requirements/base.txt (line 3))
Using cached numpy-2.2.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (62 kB)
Requirement already satisfied: networkx>=3.3 in /usr/local/lib/python3.10/dist-packages (from -r requirements/base.txt (line 4)) (3.4.2)
Requirement already satisfied: optree>=0.12.1 in /usr/local/lib/python3.10/dist-packages (from -r requirements/base.txt (line 5)) (0.17.0)
Collecting opt_einsum>=3.3.0 (from -r requirements/base.txt (line 6))
Downloading opt_einsum-3.4.0-py3-none-any.whl.metadata (6.3 kB)
Requirement already satisfied: mpmath<1.4.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements/base.txt (line 8)) (1.3.0)
Collecting dill>=0.3.8 (from -r requirements/base.txt (line 10))
Downloading dill-0.4.0-py3-none-any.whl.metadata (10 kB)
Requirement already satisfied: coverage~=7.9.1 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 1)) (7.9.2)
Requirement already satisfied: pytest==8.3.5 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 2)) (8.3.5)
Requirement already satisfied: pytest-benchmark==5.1.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 3)) (5.1.0)
Requirement already satisfied: pytest-timeout==2.4.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 4)) (2.4.0)
Requirement already satisfied: pytest-cov==6.2.1 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 5)) (6.2.1)
Requirement already satisfied: pytest-xdist==3.7.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 6)) (3.7.0)
Requirement already satisfied: pytest-random-order==1.2.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 7)) (1.2.0)
Requirement already satisfied: pytest-timestamper==0.0.10 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 8)) (0.0.10)
Requirement already satisfied: graphviz==0.21 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 9)) (0.21)
Requirement already satisfied: fdm==0.5.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 10)) (0.5.0)
Requirement already satisfied: expecttest==0.3.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 11)) (0.3.0)
Requirement already satisfied: hypothesis~=6.136.6 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 12)) (6.136.9)
Requirement already satisfied: einops in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 14)) (0.8.1)
Requirement already satisfied: litgpt==0.5.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 15)) (0.5.0)
Requirement already satisfied: absl-py in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 16)) (2.3.1)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 17)) (2.3.2)
Requirement already satisfied: xlsxwriter in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 18)) (3.2.5)
Requirement already satisfied: jsonargparse in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 19)) (4.32.1)
Collecting jsonargparse (from -r requirements/test.txt (line 19))
Using cached jsonargparse-4.40.2-py3-none-any.whl.metadata (13 kB)
Requirement already satisfied: bitsandbytes==0.46.1 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 20)) (0.46.1)
Requirement already satisfied: transformers==4.52.4 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 22)) (4.52.4)
Requirement already satisfied: diffusers==0.34.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 23)) (0.34.0)
Requirement already satisfied: accelerate in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 24)) (1.10.1)
Requirement already satisfied: asv>=0.6.4 in /usr/local/lib/python3.10/dist-packages (from -r requirements/test.txt (line 27)) (0.6.4)
Requirement already satisfied: botocore in /usr/local/lib/python3.10/dist-packages (from asvdb@ git+https://github.com/rapidsai/asvdb.git->-r requirements/test.txt (line 26)) (1.40.18)
Requirement already satisfied: boto3 in /usr/local/lib/python3.10/dist-packages (from asvdb@ git+https://github.com/rapidsai/asvdb.git->-r requirements/test.txt (line 26)) (1.40.18)
Requirement already satisfied: exceptiongroup>=1.0.0rc8 in /usr/local/lib/python3.10/dist-packages (from pytest==8.3.5->-r requirements/test.txt (line 2)) (1.3.0)
Requirement already satisfied: iniconfig in /usr/local/lib/python3.10/dist-packages (from pytest==8.3.5->-r requirements/test.txt (line 2)) (2.1.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from pytest==8.3.5->-r requirements/test.txt (line 2)) (24.2)
Requirement already satisfied: pluggy<2,>=1.5 in /usr/local/lib/python3.10/dist-packages (from pytest==8.3.5->-r requirements/test.txt (line 2)) (1.6.0)
Requirement already satisfied: tomli>=1 in /usr/local/lib/python3.10/dist-packages (from pytest==8.3.5->-r requirements/test.txt (line 2)) (2.2.1)
Requirement already satisfied: py-cpuinfo in /usr/local/lib/python3.10/dist-packages (from pytest-benchmark==5.1.0->-r requirements/test.txt (line 3)) (9.0.0)
Requirement already satisfied: execnet>=2.1 in /usr/local/lib/python3.10/dist-packages (from pytest-xdist==3.7.0->-r requirements/test.txt (line 6)) (2.1.1)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from fdm==0.5.0->-r requirements/test.txt (line 10)) (1.14.0)
Requirement already satisfied: torch>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from litgpt==0.5.0->-r requirements/test.txt (line 15)) (2.9.0a0+git3d40642)
Requirement already satisfied: lightning==2.4.0 in /usr/local/lib/python3.10/dist-packages (from litgpt==0.5.0->-r requirements/test.txt (line 15)) (2.4.0)
Requirement already satisfied: huggingface-hub>=0.23.5 in /usr/local/lib/python3.10/dist-packages (from litgpt==0.5.0->-r requirements/test.txt (line 15)) (0.34.4)
Requirement already satisfied: safetensors>=0.4.3 in /usr/local/lib/python3.10/dist-packages (from litgpt==0.5.0->-r requirements/test.txt (line 15)) (0.6.2)
Requirement already satisfied: tokenizers>=0.15.2 in /usr/local/lib/python3.10/dist-packages (from litgpt==0.5.0->-r requirements/test.txt (line 15)) (0.21.4)
Requirement already satisfied: tqdm>=4.66.0 in /usr/local/lib/python3.10/dist-packages (from litgpt==0.5.0->-r requirements/test.txt (line 15)) (4.67.1)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.52.4->-r requirements/test.txt (line 22)) (3.19.1)
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.52.4->-r requirements/test.txt (line 22)) (6.0.2)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.52.4->-r requirements/test.txt (line 22)) (2025.7.34)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers==4.52.4->-r requirements/test.txt (line 22)) (2.32.5)
Requirement already satisfied: importlib_metadata in /usr/local/lib/python3.10/dist-packages (from diffusers==0.34.0->-r requirements/test.txt (line 23)) (8.7.0)
Requirement already satisfied: Pillow in /usr/local/lib/python3.10/dist-packages (from diffusers==0.34.0->-r requirements/test.txt (line 23)) (11.3.0)
Requirement already satisfied: fsspec<2026.0,>=2022.5.0 in /usr/local/lib/python3.10/dist-packages (from fsspec[http]<2026.0,>=2022.5.0->lightning==2.4.0->litgpt==0.5.0->-r requirements/test.txt (line
↪ 15)) (2025.7.0)
Requirement already satisfied: torchmetrics<3.0,>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from lightning==2.4.0->litgpt==0.5.0->-r requirements/test.txt (line 15)) (1.8.1)
Requirement already satisfied: typing-extensions<6.0,>=4.4.0 in /usr/local/lib/python3.10/dist-packages (from lightning==2.4.0->litgpt==0.5.0->-r requirements/test.txt (line 15)) (4.15.0)
Requirement already satisfied: pytorch-lightning in /usr/local/lib/python3.10/dist-packages (from lightning==2.4.0->litgpt==0.5.0->-r requirements/test.txt (line 15)) (2.5.3)
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from lightning-utilities>0.7.0->-r requirements/base.txt (line 2)) (79.0.1)
Requirement already satisfied: attrs>=22.2.0 in /usr/local/lib/python3.10/dist-packages (from hypothesis~=6.136.6->-r requirements/test.txt (line 12)) (25.3.0)
Requirement already satisfied: sortedcontainers<3.0.0,>=2.1.0 in /usr/local/lib/python3.10/dist-packages (from hypothesis~=6.136.6->-r requirements/test.txt (line 12)) (2.4.0)
Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /usr/local/lib/python3.10/dist-packages (from fsspec[http]<2026.0,>=2022.5.0->lightning==2.4.0->litgpt==0.5.0->-r requirements/test.txt
↪ (line 15)) (3.12.15)
Requirement already satisfied: hf-xet<2.0.0,>=1.1.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.23.5->litgpt==0.5.0->-r requirements/test.txt (line 15)) (1.1.8)
Requirement already satisfied: docstring-parser>=0.15 in /usr/local/lib/python3.10/dist-packages (from jsonargparse[signatures]<=4.32.1,>=4.30.1->litgpt==0.5.0->-r requirements/test.txt (line 15))
↪ (0.17.0)
Requirement already satisfied: typeshed-client>=2.1.0 in /usr/local/lib/python3.10/dist-packages (from jsonargparse[signatures]<=4.32.1,>=4.30.1->litgpt==0.5.0->-r requirements/test.txt (line 15))
↪ (2.8.2)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=2.2.0->litgpt==0.5.0->-r requirements/test.txt (line 15)) (3.1.6)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->-r requirements/test.txt (line 17)) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->-r requirements/test.txt (line 17)) (2025.2)
Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.10/dist-packages (from pandas->-r requirements/test.txt (line 17)) (2025.2)
Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate->-r requirements/test.txt (line 24)) (7.0.0)
Requirement already satisfied: asv-runner>=v0.2.1 in /usr/local/lib/python3.10/dist-packages (from asv>=0.6.4->-r requirements/test.txt (line 27)) (0.2.1)
Requirement already satisfied: json5 in /usr/local/lib/python3.10/dist-packages (from asv>=0.6.4->-r requirements/test.txt (line 27)) (0.12.1)
Requirement already satisfied: build in /usr/local/lib/python3.10/dist-packages (from asv>=0.6.4->-r requirements/test.txt (line 27)) (1.3.0)
Requirement already satisfied: tabulate in /usr/local/lib/python3.10/dist-packages (from asv>=0.6.4->-r requirements/test.txt (line 27)) (0.9.0)
Requirement already satisfied: virtualenv in /usr/local/lib/python3.10/dist-packages (from asv>=0.6.4->-r requirements/test.txt (line 27)) (20.34.0)
Requirement already satisfied: pympler in /usr/local/lib/python3.10/dist-packages (from asv>=0.6.4->-r requirements/test.txt (line 27)) (1.1)
Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2026.0,>=2022.5.0->lightning==2.4.0->litgpt==0.5.0->-r
↪ requirements/test.txt (line 15)) (2.6.1)
Requirement already satisfied: aiosignal>=1.4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2026.0,>=2022.5.0->lightning==2.4.0->litgpt==0.5.0->-r
↪ requirements/test.txt (line 15)) (1.4.0)
Requirement already satisfied: async-timeout<6.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2026.0,>=2022.5.0->lightning==2.4.0->litgpt==0.5.0->-r
↪ requirements/test.txt (line 15)) (5.0.1)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2026.0,>=2022.5.0->lightning==2.4.0->litgpt==0.5.0->-r
↪ requirements/test.txt (line 15)) (1.7.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2026.0,>=2022.5.0->lightning==2.4.0->litgpt==0.5.0->-r
↪ requirements/test.txt (line 15)) (6.6.4)
Requirement already satisfied: propcache>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2026.0,>=2022.5.0->lightning==2.4.0->litgpt==0.5.0->-r
↪ requirements/test.txt (line 15)) (0.3.2)
Requirement already satisfied: yarl<2.0,>=1.17.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2026.0,>=2022.5.0->lightning==2.4.0->litgpt==0.5.0->-r
↪ requirements/test.txt (line 15)) (1.20.1)
Requirement already satisfied: idna>=2.0 in /usr/local/lib/python3.10/dist-packages (from
↪ yarl<2.0,>=1.17.0->aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2026.0,>=2022.5.0->lightning==2.4.0->litgpt==0.5.0->-r requirements/test.txt (line 15)) (3.10)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.8.2->pandas->-r requirements/test.txt (line 17)) (1.16.0)
Requirement already satisfied: importlib_resources>=1.4.0 in /usr/local/lib/python3.10/dist-packages (from typeshed-client>=2.1.0->jsonargparse[signatures]<=4.32.1,>=4.30.1->litgpt==0.5.0->-r
↪ requirements/test.txt (line 15)) (6.5.2)
Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from boto3->asvdb@ git+https://github.com/rapidsai/asvdb.git->-r requirements/test.txt (line 26))
↪ (1.0.1)
Requirement already satisfied: s3transfer<0.14.0,>=0.13.0 in /usr/local/lib/python3.10/dist-packages (from boto3->asvdb@ git+https://github.com/rapidsai/asvdb.git->-r requirements/test.txt (line 26))
↪ (0.13.1)
Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in /usr/local/lib/python3.10/dist-packages (from botocore->asvdb@ git+https://github.com/rapidsai/asvdb.git->-r requirements/test.txt (line
↪ 26)) (2.5.0)
Requirement already satisfied: pyproject_hooks in /usr/local/lib/python3.10/dist-packages (from build->asv>=0.6.4->-r requirements/test.txt (line 27)) (1.2.0)
Requirement already satisfied: zipp>=3.20 in /usr/local/lib/python3.10/dist-packages (from importlib_metadata->diffusers==0.34.0->-r requirements/test.txt (line 23)) (3.23.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=2.2.0->litgpt==0.5.0->-r requirements/test.txt (line 15)) (3.0.2)
Requirement already satisfied: charset_normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.52.4->-r requirements/test.txt (line 22)) (3.4.3)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.52.4->-r requirements/test.txt (line 22)) (2025.8.3)
Requirement already satisfied: distlib<1,>=0.3.7 in /usr/local/lib/python3.10/dist-packages (from virtualenv->asv>=0.6.4->-r requirements/test.txt (line 27)) (0.4.0)
Requirement already satisfied: platformdirs<5,>=3.9.1 in /usr/local/lib/python3.10/dist-packages (from virtualenv->asv>=0.6.4->-r requirements/test.txt (line 27)) (4.4.0)
Downloading looseversion-1.3.0-py2.py3-none-any.whl (8.2 kB)
Downloading opt_einsum-3.4.0-py3-none-any.whl (71 kB)
Downloading dill-0.4.0-py3-none-any.whl (119 kB)
Installing collected packages: looseversion, opt_einsum, dill
Successfully installed dill-0.4.0 looseversion-1.3.0 opt_einsum-3.4.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a
↪ virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
+ curl -Os https://uploader.codecov.io/latest/linux/codecov
+ chmod +x codecov
+ python setup.py develop
running develop
/usr/local/lib/python3.10/dist-packages/setuptools/command/develop.py:41: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!
********************************************************************************
Please avoid running ``setup.py`` and ``easy_install``.
Instead, use pypa/build, pypa/installer or other
standards-based tools.
See https://github.com/pypa/setuptools/issues/917 for details.
********************************************************************************
!!
easy_install.initialize_options(self)
/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py:90: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!
********************************************************************************
Please avoid running ``setup.py`` directly.
Instead, use pypa/build, pypa/installer or other
standards-based tools.
See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
********************************************************************************
!!
self.initialize_options()
running egg_info
creating lightning_thunder.egg-info
writing lightning_thunder.egg-info/PKG-INFO
writing dependency_links to lightning_thunder.egg-info/dependency_links.txt
writing requirements to lightning_thunder.egg-info/requires.txt
writing top-level names to lightning_thunder.egg-info/top_level.txt
writing manifest file 'lightning_thunder.egg-info/SOURCES.txt'
reading manifest file 'lightning_thunder.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*.py[cod]' found under directory '__pycache__'
warning: no previously-included files matching '*.orig' found under directory '__pycache__'
warning: no files found matching '*.md' under directory 'pl_sandbox'
warning: no previously-included files found matching '*.svg'
warning: no previously-included files found matching '*.yml'
warning: no previously-included files found matching '*.yaml'
warning: no previously-included files matching '*' found under directory 'site'
warning: no previously-included files found matching 'thunder/tests'
warning: no previously-included files matching '*' found under directory 'docs'
warning: no previously-included files found matching 'docs'
warning: no previously-included files found matching 'Makefile'
no previously-included directories found matching '.git'
no previously-included directories found matching '.github'
no previously-included directories found matching 'notebook*'
no previously-included directories found matching 'temp*'
no previously-included directories found matching 'test*'
no previously-included directories found matching 'benchmark*'
no previously-included directories found matching 'examples*'
adding license file 'LICENSE'
writing manifest file 'lightning_thunder.egg-info/SOURCES.txt'
running build_ext
Creating /usr/local/lib/python3.10/dist-packages/lightning-thunder.egg-link (link to .)
Adding lightning-thunder 0.2.5.dev0 to easy-install.pth file
Installed /workspace
Processing dependencies for lightning-thunder==0.2.5.dev0
Searching for dill==0.4.0
Best match: dill 0.4.0
Adding dill 0.4.0 to easy-install.pth file
Using /usr/local/lib/python3.10/dist-packages
Searching for mpmath==1.3.0
Best match: mpmath 1.3.0
Adding mpmath 1.3.0 to easy-install.pth file
Using /usr/local/lib/python3.10/dist-packages
Searching for opt-einsum==3.4.0
Best match: opt-einsum 3.4.0
Adding opt-einsum 3.4.0 to easy-install.pth file
Using /usr/local/lib/python3.10/dist-packages
Searching for optree==0.17.0
Best match: optree 0.17.0
Adding optree 0.17.0 to easy-install.pth file
Using /usr/local/lib/python3.10/dist-packages
Searching for networkx==3.4.2
Best match: networkx 3.4.2
Adding networkx 3.4.2 to easy-install.pth file
Using /usr/local/lib/python3.10/dist-packages
Searching for numpy==1.26.4
Best match: numpy 1.26.4
Adding numpy 1.26.4 to easy-install.pth file
Installing f2py script to /usr/local/bin
Using /usr/local/lib/python3.10/dist-packages
Searching for lightning-utilities==0.15.2
Best match: lightning-utilities 0.15.2
Adding lightning-utilities 0.15.2 to easy-install.pth file
Using /usr/local/lib/python3.10/dist-packages
Searching for looseversion==1.3.0
Best match: looseversion 1.3.0
Adding looseversion 1.3.0 to easy-install.pth file
Using /usr/local/lib/python3.10/dist-packages
Searching for typing-extensions==4.15.0
Best match: typing-extensions 4.15.0
Adding typing-extensions 4.15.0 to easy-install.pth file
Using /usr/local/lib/python3.10/dist-packages
Searching for setuptools==79.0.1
Best match: setuptools 79.0.1
Adding setuptools 79.0.1 to easy-install.pth file
Using /usr/local/lib/python3.10/dist-packages
Searching for packaging==24.2
Best match: packaging 24.2
Adding packaging 24.2 to easy-install.pth file
Using /usr/local/lib/python3.10/dist-packages/setuptools/_vendor
Finished processing dependencies for lightning-thunder==0.2.5.dev0
+ bash scripts/sanity-check.sh
+ pip list
Package Version Editable project location
--------------------- ------------------ -------------------------
absl-py 2.3.1
accelerate 1.10.1
aiohappyeyeballs 2.6.1
aiohttp 3.12.15
aiosignal 1.4.0
apex 0.1
asv 0.6.4
asv_runner 0.2.1
asvdb 0.4.2
async-timeout 5.0.1
attrs 25.3.0
autocommand 2.2.2
backports.tarfile 1.2.0
bitsandbytes 0.46.1
blinker 1.7.0
boto3 1.40.18
botocore 1.40.18
build 1.3.0
certifi 2025.8.3
charset-normalizer 3.4.3
cmake 3.31.6
coverage 7.9.2
cryptography 41.0.7
dbus-python 1.3.2
diffusers 0.34.0
dill 0.4.0
distlib 0.4.0
distro 1.9.0
docstring_parser 0.17.0
einops 0.8.1
exceptiongroup 1.3.0
execnet 2.1.1
expecttest 0.3.0
fdm 0.5.0
filelock 3.19.1
frozenlist 1.7.0
fsspec 2025.7.0
graphviz 0.21
hf-xet 1.1.8
httplib2 0.20.4
huggingface-hub 0.34.4
hypothesis 6.136.9
idna 3.10
importlib_metadata 8.7.0
importlib_resources 6.5.2
inflect 7.3.1
iniconfig 2.1.0
jaraco.collections 5.1.0
jaraco.context 5.3.0
jaraco.functools 4.0.1
jaraco.text 3.12.1
Jinja2 3.1.6
jmespath 1.0.1
json5 0.12.1
jsonargparse 4.32.1
launchpadlib 1.11.0
lazr.restfulclient 0.14.6
lazr.uri 1.0.6
lightning 2.4.0
lightning-thunder 0.2.5.dev0 /workspace
lightning-utilities 0.15.2
lintrunner 0.12.7
litgpt 0.5.0
looseversion 1.3.0
MarkupSafe 3.0.2
more-itertools 10.3.0
mpmath 1.3.0
multidict 6.6.4
networkx 3.4.2
ninja 1.13.0
numpy 1.26.4
nvfuser 0.2.30+git6d4256e
nvidia-cudnn-frontend 1.10.0
oauthlib 3.2.2
opt_einsum 3.4.0
optree 0.17.0
packaging 24.2
pandas 2.3.2
pillow 11.3.0
pip 25.2
platformdirs 4.4.0
pluggy 1.6.0
propcache 0.3.2
psutil 7.0.0
py-cpuinfo 9.0.0
pybind11 3.0.1
pybind11-global 3.0.1
PyGObject 3.48.2
PyJWT 2.7.0
Pympler 1.1
pyparsing 3.1.1
pyproject_hooks 1.2.0
pytest 8.3.5
pytest-benchmark 5.1.0
pytest-cov 6.2.1
pytest-random-order 1.2.0
pytest-timeout 2.4.0
pytest-timestamper 0.0.10
pytest-xdist 3.7.0
python-apt 2.7.7+ubuntu5
python-dateutil 2.9.0.post0
pytorch-lightning 2.5.3
pytorch-triton 3.4.0+gitf7888497
pytz 2025.2
PyYAML 6.0.2
regex 2025.7.34
requests 2.32.5
s3transfer 0.13.1
safetensors 0.6.2
setuptools 79.0.1
six 1.16.0
sortedcontainers 2.4.0
sympy 1.14.0
tabulate 0.9.0
tokenizers 0.21.4
tomli 2.2.1
torch 2.9.0a0+git3d40642
torchmetrics 1.8.1
tqdm 4.67.1
transformers 4.52.4
typeguard 4.3.0
typeshed_client 2.8.2
typing_extensions 4.15.0
tzdata 2025.2
urllib3 2.5.0
uv 0.8.13
virtualenv 20.34.0
wadllib 1.3.6
wheel 0.45.1
xlsxwriter 3.2.5
yarl 1.20.1
zipp 3.23.0
+ python -c 'import torch ; assert torch.cuda.is_available(), '\''missing GPU support!'\'''
+ python -c 'import torch ; v = torch.__version__ ; assert str(v).startswith('\''2'\''), v'
+ python -c 'from thunder.executors import nvfuser_available ; assert nvfuser_available(), '\''nvFuser is missing!'\'''
/usr/local/lib/python3.10/dist-packages/torch/cuda/__init__.py:283: UserWarning:
Found GPU0 NVIDIA L4 which is of cuda capability 8.9.
Minimum and Maximum cuda capability supported by this version of PyTorch is
(8.0) - (8.0)
warnings.warn(
/usr/local/lib/python3.10/dist-packages/torch/cuda/__init__.py:304: UserWarning:
Please install PyTorch with a following CUDA
configurations: 12.6 12.8 12.9 following instructions at
https://pytorch.org/get-started/locally/
warnings.warn(matched_cuda_warn.format(matched_arches))
/usr/local/lib/python3.10/dist-packages/torch/cuda/__init__.py:283: UserWarning:
Found GPU1 NVIDIA L4 which is of cuda capability 8.9.
Minimum and Maximum cuda capability supported by this version of PyTorch is
(8.0) - (8.0)
warnings.warn(
/usr/local/lib/python3.10/dist-packages/nvfuser_direct/__init__.py:11: UserWarning: Be careful! You've imported nvfuser_direct when the nvfuser module is already imported.
warnings.warn(
/workspace/thunder/executors/transformer_engine_v2ex.py:25: UserWarning: transformer_engine module not found!
warnings.warn("transformer_engine module not found!")
+ python -c 'from thunder.executors.triton_utils import triton_version ; assert triton_version() is not None, '\''triton is missing!'\'''
/usr/local/lib/python3.10/dist-packages/torch/cuda/__init__.py:283: UserWarning:
Found GPU0 NVIDIA L4 which is of cuda capability 8.9.
Minimum and Maximum cuda capability supported by this version of PyTorch is
(8.0) - (8.0)
warnings.warn(
/usr/local/lib/python3.10/dist-packages/torch/cuda/__init__.py:304: UserWarning:
Please install PyTorch with a following CUDA
configurations: 12.6 12.8 12.9 following instructions at
https://pytorch.org/get-started/locally/
warnings.warn(matched_cuda_warn.format(matched_arches))
/usr/local/lib/python3.10/dist-packages/torch/cuda/__init__.py:283: UserWarning:
Found GPU1 NVIDIA L4 which is of cuda capability 8.9.
Minimum and Maximum cuda capability supported by this version of PyTorch is
(8.0) - (8.0)
warnings.warn(
/usr/local/lib/python3.10/dist-packages/nvfuser_direct/__init__.py:11: UserWarning: Be careful! You've imported nvfuser_direct when the nvfuser module is already imported.
warnings.warn(
/workspace/thunder/executors/transformer_engine_v2ex.py:25: UserWarning: transformer_engine module not found!
warnings.warn("transformer_engine module not found!")
+ '[' distributed == main ']'
+ '[' distributed == ops ']'
+ '[' distributed == grads ']'
+ '[' distributed == distributed ']'
+ pytest thunder/tests/distributed -v --durations=0 --random-order-seed=42
============================= test session starts ==============================
platform linux -- Python 3.10.18, pytest-8.3.5, pluggy-1.6.0 -- /usr/bin/python3.10
cachedir: .pytest_cache
Using --random-order-bucket=module
Using --random-order-seed=42
benchmark: 5.1.0 (defaults: timer=torch.utils.benchmark.utils.timer.timer disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=True warmup_iterations=100000)
hypothesis profile 'ci' -> database=None, deadline=None, print_blob=True, derandomize=True, suppress_health_check=(HealthCheck.too_slow,)
rootdir: /workspace
configfile: pyproject.toml
plugins: timeout-2.4.0, cov-6.2.1, timestamper-0.0.10, random-order-1.2.0, benchmark-5.1.0, hypothesis-6.136.9, xdist-3.7.0, typeguard-4.3.0
timeout: 900.0s
timeout method: signal
timeout func_only: False
collecting ... collected 134 items
[2025-08-28 13:15:10] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_reduce_scatter_executor_torch_dim0_inplace_False PASSED [ 0%]
[2025-08-28 13:15:18] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_reduce_scatter_executor_nvfuser_dim_1_inplace_True PASSED [ 1%]
[2025-08-28 13:15:25] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_reduce_scatter_executor_nvfuser_dim_0_inplace_False PASSED [ 2%]
[2025-08-28 13:15:33] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_broadcast_executor_torch PASSED [ 2%]
[2025-08-28 13:15:39] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_reduce_scatter_executor_torch_dim_0_inplace_True PASSED [ 3%]
[2025-08-28 13:15:46] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_gather_executor_nvfuser_dim_1_inplace_True PASSED [ 4%]
[2025-08-28 13:15:53] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_gather_executor_torch_dim_1_inplace_True PASSED [ 5%]
[2025-08-28 13:15:59] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_native_collective_comms_executor_torch_op_all_gather_into_tensor PASSED [ 5%]
[2025-08-28 13:16:05] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_gather_executor_nvfuser_dim_0_inplace_False PASSED [ 6%]
[2025-08-28 13:16:12] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_reduce_executor_nvfuser_inplace_True PASSED [ 7%]
[2025-08-28 13:16:19] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_broadcast_executor_nvfuser PASSED [ 8%]
[2025-08-28 13:16:26] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_reduce_scatter_executor_nvfuser_dim_1_inplace_False PASSED [ 8%]
[2025-08-28 13:16:33] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_reduce_executor_nvfuser_inplace_False PASSED [ 9%]
[2025-08-28 13:16:39] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_gather_executor_nvfuser_dim6_inplace_False PASSED [ 10%]
[2025-08-28 13:16:48] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_reduce_executor_torch_inplace_True PASSED [ 11%]
[2025-08-28 13:16:54] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_gather_executor_torch_dim_1_inplace_False PASSED [ 11%]
[2025-08-28 13:17:01] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_gather_executor_torch_dim_0_inplace_False PASSED [ 12%]
[2025-08-28 13:17:07] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_native_collective_comms_executor_torch_op_reduce_scatter_tensor PASSED [ 13%]
[2025-08-28 13:17:14] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_gather_executor_torch_dim1_inplace_True PASSED [ 14%]
[2025-08-28 13:17:20] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_reduce_scatter_executor_nvfuser_dim7_inplace_True PASSED [ 14%]
[2025-08-28 13:17:27] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_reduce_scatter_executor_nvfuser_dim6_inplace_False PASSED [ 15%]
[2025-08-28 13:17:34] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_gather_executor_torch_dim_0_inplace_True PASSED [ 16%]
[2025-08-28 13:17:41] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_reduce_scatter_executor_nvfuser_dim_0_inplace_True PASSED [ 17%]
[2025-08-28 13:17:48] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_gather_executor_torch_dim0_inplace_False PASSED [ 17%]
[2025-08-28 13:17:54] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_gather_executor_nvfuser_dim_1_inplace_False PASSED [ 18%]
[2025-08-28 13:18:01] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_reduce_executor_torch_inplace_False PASSED [ 19%]
[2025-08-28 13:18:07] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_native_collective_comms_executor_nvfuser_op_reduce_scatter_tensor PASSED [ 20%]
[2025-08-28 13:18:14] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_reduce_scatter_executor_torch_dim_1_inplace_True PASSED [ 20%]
[2025-08-28 13:18:22] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_gather_executor_nvfuser_dim7_inplace_True PASSED [ 21%]
[2025-08-28 13:18:29] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_reduce_scatter_executor_torch_dim_1_inplace_False PASSED [ 22%]
[2025-08-28 13:18:36] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_gather_executor_nvfuser_dim_0_inplace_True PASSED [ 23%]
[2025-08-28 13:18:43] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_native_collective_comms_executor_nvfuser_op_all_gather_into_tensor PASSED [ 23%]
[2025-08-28 13:18:49] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_reduce_scatter_executor_torch_dim1_inplace_True PASSED [ 24%]
[2025-08-28 13:18:56] thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_reduce_scatter_executor_torch_dim_0_inplace_False PASSED [ 25%]
[2025-08-28 13:19:03] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_grad_bucketing_executor_torch_bucket_size_in_mb_0 XFAIL [ 26%]
[2025-08-28 13:19:10] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_weight_sharing PASSED [ 26%]
[2025-08-28 13:19:17] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_nvfuser_bucket_size_in_mb_0_dataset_size_2 PASSED [ 27%]
[2025-08-28 13:19:31] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_nvfuser_bucket_size_in_mb_25_dataset_size_2 PASSED [ 28%]
[2025-08-28 13:19:46] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_nvfuser_bucket_size_in_mb_0_dataset_size_1 PASSED [ 29%]
[2025-08-28 13:19:59] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_grad_parity_with_without_bucketing_executor_torch PASSED [ 29%]
[2025-08-28 13:20:08] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_torch_bucket_size_in_mb_0_dataset_size_1 PASSED [ 30%]
[2025-08-28 13:20:18] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_compile_module PASSED [ 31%]
[2025-08-28 13:20:27] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_torch_bucket_size_in_mb_25_dataset_size_2 PASSED [ 32%]
[2025-08-28 13:20:38] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_grad_bucketing_executor_torch_bucket_size_in_mb_1000 XFAIL [ 32%]
[2025-08-28 13:20:45] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_torch_bucket_size_in_mb_25_dataset_size_1 PASSED [ 33%]
[2025-08-28 13:20:56] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_grad_bucketing_executor_nvfuser_bucket_size_in_mb_0 XFAIL [ 34%]
[2025-08-28 13:21:04] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_torch_bucket_size_in_mb_0_dataset_size_2 PASSED [ 35%]
[2025-08-28 13:21:16] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_nvfuser_bucket_size_in_mb_25_dataset_size_1 PASSED [ 35%]
[2025-08-28 13:21:30] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_grad_bucketing_executor_nvfuser_bucket_size_in_mb_1000 XFAIL [ 36%]
[2025-08-28 13:21:38] thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_grad_parity_with_without_bucketing_executor_nvfuser PASSED [ 37%]
[2025-08-28 13:21:48] thunder/tests/distributed/test_ddp.py::DDPTest::test_compile_ddp_module PASSED [ 38%]
[2025-08-28 13:21:54] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_shard_unshard PASSED [ 38%]
[2025-08-28 13:22:00] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_no_sync_grad_accumulation_executor_nvfuser_bucketing_block_zero3 XFAIL [ 39%]
[2025-08-28 13:22:11] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdpv2_with_1layer_llama_meta_init PASSED [ 40%]
[2025-08-28 13:22:35] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_broadcast_from PASSED [ 41%]
[2025-08-28 13:22:42] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_limit_in_flight_allgathers_executor_nvfuser_bucketing_none_zero3 PASSED [ 41%]
[2025-08-28 13:23:02] thunder/tests/distributed/test_fsdp.py::FSDPDDPHybridTest::test_fsdp_ddp_hybrid SKIPPED [ 42%]
[2025-08-28 13:23:02] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_padding_bucketing_none_zero3 PASSED [ 43%]
[2025-08-28 13:23:10] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_limit_in_flight_allgathers_executor_nvfuser_bucketing_block_zero3 XFAIL [ 44%]
[2025-08-28 13:23:28] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_materialize_meta_tensors PASSED [ 44%]
[2025-08-28 13:23:34] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_load_original_state_dict PASSED [ 45%]
[2025-08-28 13:23:42] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_torch_bucketing_layer_zero2 XFAIL [ 46%]
[2025-08-28 13:23:50] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_no_sync_grad_accumulation_executor_torch_bucketing_block_zero2 XFAIL [ 47%]
[2025-08-28 13:23:59] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_limit_in_flight_allgathers_executor_torch_bucketing_layer_zero3 XFAIL [ 47%]
[2025-08-28 13:24:09] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_weight_sharing PASSED [ 48%]
[2025-08-28 13:24:17] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_no_sync_grad_accumulation_executor_nvfuser_bucketing_block_zero2 XFAIL [ 49%]
[2025-08-28 13:24:27] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_nvfuser_bucketing_layer_zero2 XFAIL [ 50%]
[2025-08-28 13:24:37] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdpv2_no_grad PASSED [ 50%]
[2025-08-28 13:24:54] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_limit_in_flight_allgathers_executor_nvfuser_bucketing_layer_zero3 XFAIL [ 51%]
[2025-08-28 13:25:12] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_sort_waits_executor_nvfuser PASSED [ 52%]
[2025-08-28 13:25:18] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_torch_bucketing_layer_zero3 XFAIL [ 52%]
[2025-08-28 13:25:26] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_padding_bucketing_block_zero3 PASSED [ 53%]
[2025-08-28 13:25:34] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_padding_bucketing_none_zero2 PASSED [ 54%]
[2025-08-28 13:25:42] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_limit_in_flight_allgathers_executor_torch_bucketing_block_zero3 XFAIL [ 55%]
[2025-08-28 13:25:54] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_padding_bucketing_block_zero2 PASSED [ 55%]
[2025-08-28 13:26:02] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_nvfuser_bucketing_block_zero3 XFAIL [ 56%]
[2025-08-28 13:26:11] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_nvfuser_bucketing_block_zero2 XFAIL [ 57%]
[2025-08-28 13:26:21] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_original_state_dict PASSED [ 58%]
[2025-08-28 13:26:28] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_torch_bucketing_block_zero2 XFAIL [ 58%]
[2025-08-28 13:26:36] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_limit_in_flight_allgathers_executor_torch_bucketing_none_zero3 PASSED [ 59%]
[2025-08-28 13:26:46] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_torch_bucketing_block_zero3 XFAIL [ 60%]
[2025-08-28 13:26:54] thunder/tests/distributed/test_fsdp.py::FSDPDDPHybridTest::test_fsdp_ddp_plugin SKIPPED [ 61%]
[2025-08-28 13:26:54] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_sort_waits_executor_torch PASSED [ 61%]
[2025-08-28 13:27:00] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_no_sync_grad_accumulation_executor_torch_bucketing_block_zero3 XFAIL [ 62%]
[2025-08-28 13:27:09] thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_nvfuser_bucketing_layer_zero3 XFAIL [ 63%]
[2025-08-28 13:27:20] thunder/tests/distributed/test_ddp.py::test_native_ddp_torch_cuda_thunder::dtypes::float32[0] PASSED [ 64%]
[2025-08-28 13:27:27] thunder/tests/distributed/test_fsdp.py::test_native_fsdp_torch_cuda_thunder::dtypes::float32[FSDPBucketingStrategy.BLOCK] PASSED [ 64%]
[2025-08-28 13:27:34] thunder/tests/distributed/test_fsdp.py::test_native_fsdp_nvfuser_cuda_thunder::dtypes::float32[FSDPBucketingStrategy.LAYER] PASSED [ 65%]
[2025-08-28 13:27:42] thunder/tests/distributed/test_ddp.py::test_native_ddp_nvfuser_cuda_thunder::dtypes::float32[0] PASSED [ 66%]
[2025-08-28 13:27:50] thunder/tests/distributed/test_fsdp.py::test_native_fsdp_torch_cuda_thunder::dtypes::float32[FSDPBucketingStrategy.LAYER] PASSED [ 67%]
[2025-08-28 13:27:57] thunder/tests/distributed/test_ddp.py::test_ddp_transformer_engine_v2_llama_sanity_torch_cuda_thunder::dtypes::float32 SKIPPED [ 67%]
[2025-08-28 13:27:57] thunder/tests/distributed/test_fsdp.py::test_fsdp_transformer_engine_torch_cuda_thunder::dtypes::float32[thunder_fsdp_strategy_and_intermediate_sharding2] SKIPPED [ 68%]
[2025-08-28 13:27:57] thunder/tests/distributed/test_ddp.py::test_ddp_transformer_engine_v2_torch_cuda_thunder::dtypes::float32 SKIPPED [ 69%]
[2025-08-28 13:27:57] thunder/tests/distributed/test_fsdp.py::test_fsdp_transformer_engine_bucketing_torch_cuda_thunder::dtypes::float32[thunder_fsdp_strategy_and_bucketing1] SKIPPED [ 70%]
[2025-08-28 13:27:57] thunder/tests/distributed/test_ddp.py::test_native_ddp_torch_cuda_thunder::dtypes::float32[25] PASSED [ 70%]
[2025-08-28 13:28:04] thunder/tests/distributed/test_fsdp.py::test_fsdp_transformer_engine_v2_torch_cuda_thunder::dtypes::float32[thunder_fsdp_strategy_and_intermediate_sharding2] SKIPPED [ 71%]
[2025-08-28 13:28:04] thunder/tests/distributed/test_fsdp.py::test_fsdp_transformer_engine_v2_torch_cuda_thunder::dtypes::float32[thunder_fsdp_strategy_and_intermediate_sharding0] SKIPPED [ 72%]
[2025-08-28 13:28:04] thunder/tests/distributed/test_fsdp.py::test_native_fsdp_nvfuser_cuda_thunder::dtypes::float32[FSDPBucketingStrategy.NONE] PASSED [ 73%]
[2025-08-28 13:28:12] thunder/tests/distributed/test_ddp.py::test_ddp_transformer_engine_llama_sanity_torch_cuda_thunder::dtypes::float32 SKIPPED [ 73%]
[2025-08-28 13:28:12] thunder/tests/distributed/test_fsdp.py::test_fsdp_transformer_engine_v2_bucketing_torch_cuda_thunder::dtypes::float32[thunder_fsdp_strategy_and_bucketing1] SKIPPED [ 74%]
[2025-08-28 13:28:12] thunder/tests/distributed/test_ddp.py::test_native_ddp_nvfuser_cuda_thunder::dtypes::float32[25] PASSED [ 75%]
[2025-08-28 13:28:20] thunder/tests/distributed/test_fsdp.py::test_fsdp_transformer_engine_torch_cuda_thunder::dtypes::float32[thunder_fsdp_strategy_and_intermediate_sharding1] SKIPPED [ 76%]
[2025-08-28 13:28:20] thunder/tests/distributed/test_ddp.py::test_ddp_transformer_engine_torch_cuda_thunder::dtypes::float32 SKIPPED [ 76%]
[2025-08-28 13:28:20] thunder/tests/distributed/test_fsdp.py::test_fsdp_transformer_engine_bucketing_torch_cuda_thunder::dtypes::float32[thunder_fsdp_strategy_and_bucketing0] SKIPPED [ 77%]
[2025-08-28 13:28:20] thunder/tests/distributed/test_fsdp.py::test_native_fsdp_torch_cuda_thunder::dtypes::float32[FSDPBucketingStrategy.NONE] PASSED [ 78%]
[2025-08-28 13:28:27] thunder/tests/distributed/test_fsdp.py::test_fsdp_transformer_engine_v2_torch_cuda_thunder::dtypes::float32[thunder_fsdp_strategy_and_intermediate_sharding1] SKIPPED [ 79%]
[2025-08-28 13:28:27] thunder/tests/distributed/test_fsdp.py::test_fsdp_transformer_engine_torch_cuda_thunder::dtypes::float32[thunder_fsdp_strategy_and_intermediate_sharding0] SKIPPED [ 79%]
[2025-08-28 13:28:27] thunder/tests/distributed/test_fsdp.py::test_fsdp_transformer_engine_v2_bucketing_torch_cuda_thunder::dtypes::float32[thunder_fsdp_strategy_and_bucketing0] SKIPPED [ 80%]
[2025-08-28 13:28:27] thunder/tests/distributed/test_fsdp.py::test_native_fsdp_nvfuser_cuda_thunder::dtypes::float32[FSDPBucketingStrategy.BLOCK] PASSED [ 81%]
[2025-08-28 13:28:35] thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_basic_op_executor_torch_fn_key_x * w PASSED [ 82%]
[2025-08-28 13:28:48] thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_basic_op_executor_nvfuser_fn_key_torch_mul PASSED [ 82%]
[2025-08-28 13:28:59] thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_unsupported_mixed_input PASSED [ 83%]
[2025-08-28 13:29:05] thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_basic_op_executor_nvfuser_fn_key_x_mul(w) PASSED [ 84%]
[2025-08-28 13:29:16] thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_basic_op_executor_nvfuser_fn_key_x * w PASSED [ 85%]
[2025-08-28 13:29:27] thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_incorrect_cotangent PASSED [ 85%]
[2025-08-28 13:29:33] thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_basic_op_executor_torch_fn_key_x_mul(w) PASSED [ 86%]
[2025-08-28 13:29:45] thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_basic_op_executor_torch_fn_key_torch_mul PASSED [ 87%]
[2025-08-28 13:29:56] thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_unsupported PASSED [ 88%]
[2025-08-28 13:30:02] thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_opinfo_reshape_nvfuser PASSED [ 88%]
[2025-08-28 13:30:12] thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_opinfo_reshape_torch PASSED [ 89%]
[2025-08-28 13:30:23] thunder/tests/distributed/test_checkpoint.py::DistributedCheckpointTest::test_get_model_state_dict PASSED [ 90%]
[2025-08-28 13:30:29] thunder/tests/distributed/test_checkpoint.py::DistributedCheckpointTest::test_load_model_state_dict PASSED [ 91%]
[2025-08-28 13:30:35] thunder/tests/distributed/test_checkpoint.py::test_split_state_dict PASSED [ 91%]
[2025-08-28 13:30:35] thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_linear_name_row_bias_False PASSED [ 92%]
[2025-08-28 13:30:43] thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_parallel_mlp_meta_init_True PASSED [ 93%]
[2025-08-28 13:30:53] thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_parallel_mlp_meta_init_False PASSED [ 94%]
[2025-08-28 13:31:02] thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_embedding_name_column PASSED [ 94%]
[2025-08-28 13:31:10] thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_embedding_name_row PASSED [ 95%]
[2025-08-28 13:31:17] thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_both_column_and_row_bias_False PASSED [ 96%]
[2025-08-28 13:31:27] thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_litgpt_causal_self_attention PASSED [ 97%]
[2025-08-28 13:31:39] thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_linear_name_column_bias_True PASSED [ 97%]
[2025-08-28 13:31:49] thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_both_column_and_row_bias_True PASSED [ 98%]
[2025-08-28 13:32:00] thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_linear_name_row_bias_True PASSED [ 99%]
[2025-08-28 13:32:09] thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_linear_name_column_bias_False PASSED [100%]
============================== slowest durations ===============================
24.25s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdpv2_with_1layer_llama_meta_init
20.24s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_limit_in_flight_allgathers_executor_nvfuser_bucketing_none_zero3
18.44s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_limit_in_flight_allgathers_executor_nvfuser_bucketing_block_zero3
17.94s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_limit_in_flight_allgathers_executor_nvfuser_bucketing_layer_zero3
17.23s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdpv2_no_grad
15.43s call thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_nvfuser_bucket_size_in_mb_25_dataset_size_2
13.93s call thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_nvfuser_bucket_size_in_mb_0_dataset_size_2
13.43s call thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_nvfuser_bucket_size_in_mb_25_dataset_size_1
13.03s call thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_nvfuser_bucket_size_in_mb_0_dataset_size_1
12.43s call thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_basic_op_executor_torch_fn_key_x * w
12.33s call thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_torch_bucket_size_in_mb_0_dataset_size_2
12.13s call thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_litgpt_causal_self_attention
11.72s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_limit_in_flight_allgathers_executor_torch_bucketing_block_zero3
11.43s call thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_opinfo_reshape_torch
11.42s call thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_torch_bucket_size_in_mb_25_dataset_size_2
11.33s call thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_basic_op_executor_torch_fn_key_torch_mul
11.32s call thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_basic_op_executor_torch_fn_key_x_mul(w)
11.32s call thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_basic_op_executor_nvfuser_fn_key_x * w
11.32s call thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_basic_op_executor_nvfuser_fn_key_x_mul(w)
11.22s call thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_basic_op_executor_nvfuser_fn_key_torch_mul
10.92s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_no_sync_grad_accumulation_executor_nvfuser_bucketing_block_zero3
10.92s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_nvfuser_bucketing_layer_zero3
10.52s call thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_linear_name_column_bias_True
10.52s call thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_both_column_and_row_bias_True
10.42s call thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_torch_bucket_size_in_mb_25_dataset_size_1
10.42s call thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_with_no_sync_grad_accumulation_executor_torch_bucket_size_in_mb_0_dataset_size_1
10.42s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_no_sync_grad_accumulation_executor_nvfuser_bucketing_block_zero2
10.02s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_limit_in_flight_allgathers_executor_torch_bucketing_none_zero3
9.92s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_limit_in_flight_allgathers_executor_torch_bucketing_layer_zero3
9.92s call thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_both_column_and_row_bias_False
9.62s call thunder/tests/distributed/test_dtensor.py::DTensorTest::test_dtensor_opinfo_reshape_nvfuser
9.42s call thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_grad_parity_with_without_bucketing_executor_nvfuser
9.42s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_nvfuser_bucketing_block_zero2
9.42s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_nvfuser_bucketing_layer_zero2
9.42s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_nvfuser_bucketing_block_zero3
9.42s call thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_parallel_mlp_meta_init_True
9.42s call thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_parallel_mlp_meta_init_False
8.92s call thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_linear_name_row_bias_True
8.92s call thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_compile_module
8.43s call thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_reduce_scatter_executor_torch_dim0_inplace_False
8.42s call thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_grad_bucketing_executor_nvfuser_bucket_size_in_mb_0
8.42s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_no_sync_grad_accumulation_executor_torch_bucketing_block_zero3
8.42s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_load_original_state_dict
8.42s call thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_grad_parity_with_without_bucketing_executor_torch
8.42s call thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_reduce_scatter_executor_torch_dim_1_inplace_True
8.32s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_no_sync_grad_accumulation_executor_torch_bucketing_block_zero2
8.32s call thunder/tests/distributed/test_ddp.py::DDPTest::test_ddp_grad_bucketing_executor_nvfuser_bucket_size_in_mb_1000
8.22s call thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_all_gather_executor_nvfuser_dim6_inplace_False
8.12s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_weight_sharing
8.10s call thunder/tests/distributed/test_ddp.py::test_native_ddp_nvfuser_cuda_thunder.dtypes.float32[0]
8.04s call thunder/tests/distributed/test_ddp.py::test_native_ddp_nvfuser_cuda_thunder.dtypes.float32[25]
8.04s call thunder/tests/distributed/test_fsdp.py::test_native_fsdp_nvfuser_cuda_thunder.dtypes.float32[FSDPBucketingStrategy.LAYER]
8.03s call thunder/tests/distributed/test_fsdp.py::test_native_fsdp_nvfuser_cuda_thunder.dtypes.float32[FSDPBucketingStrategy.BLOCK]
8.02s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_padding_bucketing_none_zero2
8.02s call thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_linear_name_row_bias_False
7.98s call thunder/tests/distributed/test_fsdp.py::test_native_fsdp_nvfuser_cuda_thunder.dtypes.float32[FSDPBucketingStrategy.NONE]
7.92s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_padding_bucketing_none_zero3
7.92s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_torch_bucketing_block_zero2
7.92s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_padding_bucketing_block_zero3
7.92s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_torch_bucketing_layer_zero2
7.92s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_torch_bucketing_block_zero3
7.92s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_with_padding_bucketing_block_zero2
7.92s call thunder/tests/distributed/test_fsdp.py::FSDPTest::test_fsdp_grad_parity_with_without_bucketing_executor_torch_bucketing_layer_zero3
7.92s call thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_linear_name_column_bias_False
7.42s call thunder/tests/distributed/test_tensor_parallel.py::TensorParallelTest::test_embedding_name_column
7.38s call thunder/tests/distributed/test_ddp.py::test_native_ddp_torch_cuda_thunder.dtypes.float32[25]
7.35s call thunder/tests/distributed/test_ddp.py::test_native_ddp_torch_cuda_thunder.dtypes.float32[0]
7.12s call thunder/tests/distributed/test_ops.py::DistributedCollectiveOpTest::test_reduce_scatter_executor_nvfuser_dim_1_inplace_True
…(truncated)
Loading