Commit d3ab6bf
fix(metax): fall back to Triton cache ops when vllm C extensions are missing
When vllm is built with VLLM_TARGET_DEVICE=empty, torch.ops._C_cache_ops
is not registered. The MetaX flash attention backend calls
vllm._custom_ops.reshape_and_cache_flash which delegates to that C
extension — causing AttributeError at runtime.
vllm already ships a pure-Triton implementation
(triton_reshape_and_cache_flash) for platforms without compiled
extensions. This patch routes _custom_ops through the Triton fallback
when _C_cache_ops is absent.
The patch only fires when _C_cache_ops is missing — standard vllm
wheels with compiled C extensions are unaffected.
Note: _C activation ops (silu_and_mul etc.) do NOT need stubs here.
The MetaX vendor backend (silu_and_mul_maca) already bypasses vllm's
SiluAndMul/GeluAndMul classes and computes the activations directly
with F.silu/F.gelu — no dependency on torch.ops._C.
Tested on MetaX C550 (MACA 3.7.2.0, torch 2.8.0+metax):
vllm 0.20.2 empty + vllm-plugin-FL + Qwen3-4B, eager + flash attn,
inference succeeds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>1 parent 1326a33 commit d3ab6bf
1 file changed
Lines changed: 24 additions & 0 deletions
Lines changed: 24 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
0 commit comments