Describe the bug
vLLM 相关的模块导入失败导致的。问题出在 test_apply_repetition_penalties.py 这个测试文件尝试导入 vLLM 时崩溃了。
- 镜像:
harbor.baai.ac.cn/flagtree/flagtree-metax-py312-torch2.8.0-vllm0.15.0-metax3.5.3.x-ubuntu22.04:202604-0.5.1
执行命令:
root@p-phy-zy-daxing-kt-muxi-mxc550-node-prod-15-253:/home/FlagGems/tests# pytest -m abs --ref cpu
报错信息如下:
`root@p-phy-zy-daxing-kt-muxi-mxc550-node-prod-15-253:/home/FlagGems/tests# pytest -m abs --ref cpu
=============================================================================== test session starts ===============================================================================
platform linux -- Python 3.12.11, pytest-8.4.1, pluggy-1.6.0
rootdir: /home/FlagGems
configfile: pytest.ini
plugins: anyio-4.12.1
collecting 365 items Fatal Python error: Aborted
Current thread 0x00007f55f693f740 (most recent call first):
File "", line 488 in _call_with_frames_removed
File "", line 1293 in create_module
File "", line 813 in module_from_spec
File "", line 921 in _load_unlocked
File "", line 1331 in _find_and_load_unlocked
File "", line 1360 in _find_and_load
File "/opt/conda/lib/python3.12/site-packages/vllm_metax/platform.py", line 194 in import_kernels
File "/opt/conda/lib/python3.12/site-packages/vllm/_custom_ops.py", line 19 in
File "", line 488 in _call_with_frames_removed
File "", line 999 in exec_module
File "", line 935 in _load_unlocked
File "", line 1331 in _find_and_load_unlocked
File "", line 1360 in _find_and_load
File "/home/FlagGems/tests/test_apply_repetition_penalties.py", line 13 in _init_vllm
File "/home/FlagGems/tests/test_apply_repetition_penalties.py", line 32 in
File "/opt/conda/lib/python3.12/site-packages/_pytest/assertion/rewrite.py", line 186 in exec_module
File "", line 935 in _load_unlocked
File "", line 1331 in _find_and_load_unlocked
File "", line 1360 in _find_and_load
File "", line 1387 in _gcd_import
File "/opt/conda/lib/python3.12/importlib/init.py", line 90 in import_module
File "/opt/conda/lib/python3.12/site-packages/_pytest/pathlib.py", line 587 in import_path
File "/opt/conda/lib/python3.12/site-packages/_pytest/python.py", line 498 in importtestmodule
File "/opt/conda/lib/python3.12/site-packages/_pytest/python.py", line 551 in _getobj
File "/opt/conda/lib/python3.12/site-packages/_pytest/python.py", line 280 in obj
File "/opt/conda/lib/python3.12/site-packages/_pytest/python.py", line 567 in _register_setup_module_fixture
File "/opt/conda/lib/python3.12/site-packages/_pytest/python.py", line 554 in collect
File "/opt/conda/lib/python3.12/site-packages/_pytest/runner.py", line 389 in collect
File "/opt/conda/lib/python3.12/site-packages/_pytest/runner.py", line 344 in from_call
File "/opt/conda/lib/python3.12/site-packages/_pytest/runner.py", line 391 in pytest_make_collect_report
File "/opt/conda/lib/python3.12/site-packages/pluggy/_callers.py", line 121 in _multicall
File "/opt/conda/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/opt/conda/lib/python3.12/site-packages/pluggy/_hooks.py", line 512 in call
File "/opt/conda/lib/python3.12/site-packages/_pytest/runner.py", line 567 in collect_one_node
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 839 in _collect_one_node
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 974 in genitems
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 979 in genitems
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 813 in perform_collect
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 353 in pytest_collection
File "/opt/conda/lib/python3.12/site-packages/pluggy/_callers.py", line 121 in _multicall
File "/opt/conda/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/opt/conda/lib/python3.12/site-packages/pluggy/_hooks.py", line 512 in call
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 342 in _main
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 289 in wrap_session
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 336 in pytest_cmdline_main
File "/opt/conda/lib/python3.12/site-packages/pluggy/_callers.py", line 121 in _multicall
File "/opt/conda/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/opt/conda/lib/python3.12/site-packages/pluggy/_hooks.py", line 512 in call
File "/opt/conda/lib/python3.12/site-packages/_pytest/config/init.py", line 175 in main
File "/opt/conda/lib/python3.12/site-packages/_pytest/config/init.py", line 201 in console_main
File "/opt/conda/bin/pytest", line 8 in
Extension modules: yaml._yaml, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, maca_utils, sqlalchemy.cyextension.collections, sqlalchemy.cyextension.immutabledict, sqlalchemy.cyextension.processors, sqlalchemy.cyextension.resultproxy, sqlalchemy.cyextension.util, greenlet._greenlet, psutil._psutil_linux, zmq.backend.cython._zmq, _brotli, regex._regex, PIL._imaging, PIL._imagingft, chardet.models, chardet.pipeline.ascii, chardet.pipeline.confusion, chardet.pipeline.escape, chardet.pipeline.magic, chardet.pipeline.statistical, chardet.pipeline.structural, chardet.pipeline.utf8, chardet.pipeline.utf1632, chardet.pipeline.validity, chardet.pipeline.orchestrator, requests.packages.chardet.models, requests.packages.chardet.pipeline.ascii, requests.packages.chardet.pipeline.confusion, requests.packages.chardet.pipeline.escape, requests.packages.chardet.pipeline.magic, requests.packages.chardet.pipeline.statistical, requests.packages.chardet.pipeline.structural, requests.packages.chardet.pipeline.utf8, requests.packages.chardet.pipeline.utf1632, requests.packages.chardet.pipeline.validity, requests.packages.chardet.pipeline.orchestrator (total: 60)
Aborted (core dumped)`
当前解决方案:卸载 pip uninstall vllm_metax
Environment details
- 镜像:
harbor.baai.ac.cn/flagtree/flagtree-metax-py312-torch2.8.0-vllm0.15.0-metax3.5.3.x-ubuntu22.04:202604-0.5.1
root@p-phy-zy-daxing-kt-muxi-mxc550-node-prod-15-253:/home/FlagGems/tests# pip list | grep vllm vllm 0.15.0 vllm_metax 0.15.0+g24fb31.d20260310.maca3.5.3.20.torch2.8
Describe the bug
vLLM 相关的模块导入失败导致的。问题出在 test_apply_repetition_penalties.py 这个测试文件尝试导入 vLLM 时崩溃了。
harbor.baai.ac.cn/flagtree/flagtree-metax-py312-torch2.8.0-vllm0.15.0-metax3.5.3.x-ubuntu22.04:202604-0.5.1执行命令:
root@p-phy-zy-daxing-kt-muxi-mxc550-node-prod-15-253:/home/FlagGems/tests# pytest -m abs --ref cpu报错信息如下:
`root@p-phy-zy-daxing-kt-muxi-mxc550-node-prod-15-253:/home/FlagGems/tests# pytest -m abs --ref cpu
=============================================================================== test session starts ===============================================================================
platform linux -- Python 3.12.11, pytest-8.4.1, pluggy-1.6.0
rootdir: /home/FlagGems
configfile: pytest.ini
plugins: anyio-4.12.1
collecting 365 items Fatal Python error: Aborted
Current thread 0x00007f55f693f740 (most recent call first):
File "", line 488 in _call_with_frames_removed
File "", line 1293 in create_module
File "", line 813 in module_from_spec
File "", line 921 in _load_unlocked
File "", line 1331 in _find_and_load_unlocked
File "", line 1360 in _find_and_load
File "/opt/conda/lib/python3.12/site-packages/vllm_metax/platform.py", line 194 in import_kernels
File "/opt/conda/lib/python3.12/site-packages/vllm/_custom_ops.py", line 19 in
File "", line 488 in _call_with_frames_removed
File "", line 999 in exec_module
File "", line 935 in _load_unlocked
File "", line 1331 in _find_and_load_unlocked
File "", line 1360 in _find_and_load
File "/home/FlagGems/tests/test_apply_repetition_penalties.py", line 13 in _init_vllm
File "/home/FlagGems/tests/test_apply_repetition_penalties.py", line 32 in
File "/opt/conda/lib/python3.12/site-packages/_pytest/assertion/rewrite.py", line 186 in exec_module
File "", line 935 in _load_unlocked
File "", line 1331 in _find_and_load_unlocked
File "", line 1360 in _find_and_load
File "", line 1387 in _gcd_import
File "/opt/conda/lib/python3.12/importlib/init.py", line 90 in import_module
File "/opt/conda/lib/python3.12/site-packages/_pytest/pathlib.py", line 587 in import_path
File "/opt/conda/lib/python3.12/site-packages/_pytest/python.py", line 498 in importtestmodule
File "/opt/conda/lib/python3.12/site-packages/_pytest/python.py", line 551 in _getobj
File "/opt/conda/lib/python3.12/site-packages/_pytest/python.py", line 280 in obj
File "/opt/conda/lib/python3.12/site-packages/_pytest/python.py", line 567 in _register_setup_module_fixture
File "/opt/conda/lib/python3.12/site-packages/_pytest/python.py", line 554 in collect
File "/opt/conda/lib/python3.12/site-packages/_pytest/runner.py", line 389 in collect
File "/opt/conda/lib/python3.12/site-packages/_pytest/runner.py", line 344 in from_call
File "/opt/conda/lib/python3.12/site-packages/_pytest/runner.py", line 391 in pytest_make_collect_report
File "/opt/conda/lib/python3.12/site-packages/pluggy/_callers.py", line 121 in _multicall
File "/opt/conda/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/opt/conda/lib/python3.12/site-packages/pluggy/_hooks.py", line 512 in call
File "/opt/conda/lib/python3.12/site-packages/_pytest/runner.py", line 567 in collect_one_node
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 839 in _collect_one_node
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 974 in genitems
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 979 in genitems
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 813 in perform_collect
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 353 in pytest_collection
File "/opt/conda/lib/python3.12/site-packages/pluggy/_callers.py", line 121 in _multicall
File "/opt/conda/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/opt/conda/lib/python3.12/site-packages/pluggy/_hooks.py", line 512 in call
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 342 in _main
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 289 in wrap_session
File "/opt/conda/lib/python3.12/site-packages/_pytest/main.py", line 336 in pytest_cmdline_main
File "/opt/conda/lib/python3.12/site-packages/pluggy/_callers.py", line 121 in _multicall
File "/opt/conda/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/opt/conda/lib/python3.12/site-packages/pluggy/_hooks.py", line 512 in call
File "/opt/conda/lib/python3.12/site-packages/_pytest/config/init.py", line 175 in main
File "/opt/conda/lib/python3.12/site-packages/_pytest/config/init.py", line 201 in console_main
File "/opt/conda/bin/pytest", line 8 in
Extension modules: yaml._yaml, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, maca_utils, sqlalchemy.cyextension.collections, sqlalchemy.cyextension.immutabledict, sqlalchemy.cyextension.processors, sqlalchemy.cyextension.resultproxy, sqlalchemy.cyextension.util, greenlet._greenlet, psutil._psutil_linux, zmq.backend.cython._zmq, _brotli, regex._regex, PIL._imaging, PIL._imagingft, chardet.models, chardet.pipeline.ascii, chardet.pipeline.confusion, chardet.pipeline.escape, chardet.pipeline.magic, chardet.pipeline.statistical, chardet.pipeline.structural, chardet.pipeline.utf8, chardet.pipeline.utf1632, chardet.pipeline.validity, chardet.pipeline.orchestrator, requests.packages.chardet.models, requests.packages.chardet.pipeline.ascii, requests.packages.chardet.pipeline.confusion, requests.packages.chardet.pipeline.escape, requests.packages.chardet.pipeline.magic, requests.packages.chardet.pipeline.statistical, requests.packages.chardet.pipeline.structural, requests.packages.chardet.pipeline.utf8, requests.packages.chardet.pipeline.utf1632, requests.packages.chardet.pipeline.validity, requests.packages.chardet.pipeline.orchestrator (total: 60)
Aborted (core dumped)`
当前解决方案:卸载 pip uninstall vllm_metax
Environment details
harbor.baai.ac.cn/flagtree/flagtree-metax-py312-torch2.8.0-vllm0.15.0-metax3.5.3.x-ubuntu22.04:202604-0.5.1root@p-phy-zy-daxing-kt-muxi-mxc550-node-prod-15-253:/home/FlagGems/tests# pip list | grep vllm vllm 0.15.0 vllm_metax 0.15.0+g24fb31.d20260310.maca3.5.3.20.torch2.8