Check CPU AVX-512 support before selecting ONNX GPU backends#303
Open
peteonrails wants to merge 1 commit intomainfrom
Open
Check CPU AVX-512 support before selecting ONNX GPU backends#303peteonrails wants to merge 1 commit intomainfrom
peteonrails wants to merge 1 commit intomainfrom
Conversation
The ONNX CUDA and ROCm binaries bundle prebuilt ONNX Runtime which contains AVX-512 instructions. On CPUs without AVX-512 (e.g., Zen 3), these binaries crash with SIGILL. The backend selector now checks /proc/cpuinfo for avx512f before recommending GPU backends, falling back to onnx-avx2 instead. Also removes "experimental" labeling from Parakeet across code and docs -- it's been stable and integrated into the setup system since v0.5.0. Closes #290
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/proc/cpuinfoforavx512fbefore recommending CUDA or ROCm binaries, which bundle ONNX Runtime with AVX-512 instructionsonnx-avx2instead of selecting a binary that will SIGILLvoxtype setup onnxstatus output when GPU is detected but CPU lacks AVX-512voxtype setup gpu --enableto explain the limitation and suggest alternativesContext
Reported by a user on Omarchy with a Ryzen 7 5700X3D (Zen 3, AVX2 only) and Radeon RX 6800. Running
voxtype setup onnx --enableauto-selected the ROCm binary, which crashed with SIGILL due to 89K AVX-512 instructions in the bundled ONNX Runtime. The user had no path to working ONNX because the selector kept picking ROCm.Test plan
voxtype setup onnx --enableshould selectonnx-avx2, notonnx-rocmvoxtype setup onnx --enableshould still selectonnx-rocmvoxtype setup onnxstatus output shows AVX-512 note when GPU detected but CPU lacks AVX-512voxtype setup gpu --enablein ONNX mode on AVX2-only CPU shows helpful error with alternativesshow-config,setup check,setup model --list