环境
| 组件 |
版本 |
| 硬件 |
Iluvatar BI-V150 × 8 |
| CoreX Driver |
4.5.0 |
| torch |
2.10.0+corex.4.5.0 |
| flagtree |
0.6.0+iluvatar3.6 (triton 3.6.0) |
| vllm |
0.24.0+empty |
| vllm-plugin-fl |
0.2.0 |
对比基准:triton 3.2.0+corex.4.5.0(无 flagtree)— 同硬件上全部测试通过。
问题 1:0 compatible backends for target (cuda)(已在 vllm-plugin-FL 侧修复)
torch._inductor 始终用 GPUTarget(backend='cuda') 构造 triton 目标,但 flagtree triton 3.6.0 注册的 backend 为 'iluvatar',且其 supports_target() 检查 target.backend == 'corex',导致:
RuntimeError: 0 compatible backends for target (cuda) ([]).
There should only be one.
已在 vllm-plugin-FL 侧修复(runtime probe supports_target() 确定正确 remap,将 'cuda' → 'corex'):
physics31415926/vllm-plugin-FL@b1e7aed
不过,从 flagtree 角度,理想的做法是让 flagtree triton 的 supports_target() 也接受 'cuda',或者让 backend 注册时同时注册 'cuda' alias,避免下游框架需要 workaround。
问题 2:TritonILUVATARGPUMMAReduceThreadLocality pass 导致 PassManager::run failed(待修复)
应用问题 1 的修复后,triton 可以正常选择 backend 并开始编译,但在编译 vllm 的 kernel_unified_attention 时,flagtree 定制 pass TritonILUVATARGPUMMAReduceThreadLocality 触发 SSA dominance 检查失败。
完整报错
vllm/v1/attention/ops/triton_unified_attention.py:179:0: error: Failures have been detected while processing an MLIR pass pipeline
vllm/v1/attention/ops/triton_unified_attention.py:179:0: note: Pipeline failed while executing [`TritonILUVATARGPUMMAReduceThreadLocality` on 'builtin.module' operation]
vllm/v1/attention/ops/triton_attention_helpers.py:415:19: error: operand #0 does not dominate this use
vllm/v1/attention/ops/triton_unified_attention.py:539:44: note: called from
vllm/v1/attention/ops/triton_attention_helpers.py:415:19: note: operand defined here (op in the same block)
File "vllm/v1/attention/ops/triton_unified_attention.py", line 1019, in unified_attention
RuntimeError: PassManager::run failed
出错代码
triton_attention_helpers.py:415:
def softmax_step(S, M, L):
m_j = tl.max(S, axis=1)
m_j = tl.where(m_j > float("-inf"), m_j, 0.0)
P = tl.exp(S - m_j[:, None])
l_j = tl.sum(P, axis=1)
alpha = tl.exp(M - m_j) # ← line 415: operand M does not dominate this use
L_new = L * alpha + l_j
return m_j, L_new, P, alpha
M 是函数入参,在 TritonILUVATARGPUMMAReduceThreadLocality pass 的 IR 变换后,M 对应的 MLIR operand 不再支配 tl.exp(M - m_j) 处的 use,触发 SSA dominance 检查失败。triton 3.2.x 的同名 pass 无此问题。
调用链
kernel_unified_attention (triton_unified_attention.py:179)
→ softmax_step (triton_unified_attention.py:539)
→ tl.exp(M - m_j) (triton_attention_helpers.py:415)
→ TritonILUVATARGPUMMAReduceThreadLocality FAIL
期望修复
- 方案 A(推荐):修复
TritonILUVATARGPUMMAReduceThreadLocality pass,使其能正确处理此类 IR 模式(函数入参在同 block 内的 use),与 triton 3.2.x 行为一致。
- 方案 B:在 pass 中对此类 SSA dominance 情况做容错处理。
复现
# 需要 flagtree 0.6.0+iluvatar3.6 环境,Iluvatar BI-V150
import os
os.environ["VLLM_ALLOW_LONG_MAX_MODEL_LEN"] = "1"
os.environ["VLLM_PLUGINS"] = "fl"
from vllm import LLM, SamplingParams
llm = LLM(
model="<any-attention-model>",
tensor_parallel_size=4,
max_model_len=4096,
gpu_memory_utilization=0.60,
enforce_eager=True,
)
outputs = llm.generate(["Hello"], SamplingParams(max_tokens=10, temperature=0.0))
环境
2.10.0+corex.4.5.00.6.0+iluvatar3.6(triton 3.6.0)0.24.0+empty0.2.0对比基准:
triton 3.2.0+corex.4.5.0(无 flagtree)— 同硬件上全部测试通过。问题 1:
0 compatible backends for target (cuda)(已在 vllm-plugin-FL 侧修复)torch._inductor始终用GPUTarget(backend='cuda')构造 triton 目标,但 flagtree triton 3.6.0 注册的 backend 为'iluvatar',且其supports_target()检查target.backend == 'corex',导致:已在 vllm-plugin-FL 侧修复(runtime probe
supports_target()确定正确 remap,将'cuda'→'corex'):physics31415926/vllm-plugin-FL@b1e7aed
不过,从 flagtree 角度,理想的做法是让 flagtree triton 的
supports_target()也接受'cuda',或者让 backend 注册时同时注册'cuda'alias,避免下游框架需要 workaround。问题 2:
TritonILUVATARGPUMMAReduceThreadLocalitypass 导致PassManager::run failed(待修复)应用问题 1 的修复后,triton 可以正常选择 backend 并开始编译,但在编译 vllm 的
kernel_unified_attention时,flagtree 定制 passTritonILUVATARGPUMMAReduceThreadLocality触发 SSA dominance 检查失败。完整报错
出错代码
triton_attention_helpers.py:415:M是函数入参,在TritonILUVATARGPUMMAReduceThreadLocalitypass 的 IR 变换后,M对应的 MLIR operand 不再支配tl.exp(M - m_j)处的 use,触发 SSA dominance 检查失败。triton 3.2.x 的同名 pass 无此问题。调用链
期望修复
TritonILUVATARGPUMMAReduceThreadLocalitypass,使其能正确处理此类 IR 模式(函数入参在同 block 内的 use),与 triton 3.2.x 行为一致。复现