Skip to content

Commit 4d11301

Browse files
authored
feat(app): app-layer env framework (env.app) + vllm flashinfer gate (#409)
* feat(app): add app-layer env framework (env.app) + vllm flashinfer gate Configs gain an env.app.{app} dimension keyed by app name, so per-app per-backend env vars (e.g. VLLM_USE_FLASHINFER_SAMPLER=0 for NVIDIA vLLM 0.24.0) have a home and a delivery channel without leaking into other app images. - configs.yaml: env.app.vllm.VLLM_USE_FLASHINFER_SAMPLER=0 for nvidia-cuda12.8/13.3 (vLLM 0.24.0 imports flashinfer unconditionally; runtime doesn't ship it, so the sampler is gated off at the app layer) - app/{vllm,megatron}/Containerfile: APP_ENV build arg baked into /etc/profile.d/app_env.sh (same pattern as runtime_env; sourced via BASH_ENV plumbing) - scripts/generate_matrix.py: --app <name> mode = runtime matrix + app_env serialized as KEY=value\n...; --runtime output unchanged (regression byte-identical) - app/vllm/Containerfile: default VLLM_VERSION 0.20.2 -> 0.24.0 and pin vllm=={version}+flagos — a bare =={version} resolves the official wheel on the fallback index and pulls a newer torch over the runtime's pinned matrix (torch-pull trap, seen live on h20) - megatron-app-image.yml: matrix via --app megatron, pass APP_ENV - docs/gen_data.py: carry env.app into docs/data/images.yaml Verified on node: vllm 0.24.0+flagos installs inert (torch stays 2.10.0+cu128), VLLM_USE_FLASHINFER_SAMPLER=0 reachable in container shell. * docs(vllm): address PR #409 review comments - configs.yaml: translate env.app.vllm comments to English - generate_matrix.py: clarify "--app: all" docstring as "all backends" - report-vllm-0.24.0.md: SDK -> MACA in TL;DR; drop the §6 container and log-file detail; delete the metax monkey-patch comparison (irrelevant to NVIDIA); document the long-term plugin-wheel resolution for the §6.1(2) toolchain gap and the appendix install block - vllm-verification-matrix.md: reword "均 Qwen3-4B E2E" to natural Chinese
1 parent ebd6085 commit 4d11301

8 files changed

Lines changed: 217 additions & 33 deletions

File tree

.github/workflows/megatron-app-image.yml

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -28,9 +28,9 @@ name: Megatron app image
2828
# runtime's torch/triton/flag_gems matrix cannot be disturbed (the repack
2929
# facility was removed 2026-08-14).
3030
#
31-
# Backend matrix comes from scripts/generate_matrix.py --runtime (same source
32-
# as runtime-image.yaml). Per-vendor PyPI (flagos_pypi) is searched first;
33-
# aliyun is the fallback.
31+
# Backend matrix comes from scripts/generate_matrix.py --app megatron (same
32+
# runtime fields as runtime-image.yaml plus the per-backend app env vars).
33+
# Per-vendor PyPI (flagos_pypi) is searched first; aliyun is the fallback.
3434

3535
on:
3636
workflow_dispatch:
@@ -84,9 +84,9 @@ jobs:
8484
shell: bash
8585
run: |
8686
if [[ "${{ inputs.backend }}" == "all" ]]; then
87-
python3 scripts/generate_matrix.py --runtime > matrix.json
87+
python3 scripts/generate_matrix.py --app megatron > matrix.json
8888
else
89-
python3 scripts/generate_matrix.py --runtime ${{ inputs.backend }} > matrix.json
89+
python3 scripts/generate_matrix.py --app megatron ${{ inputs.backend }} > matrix.json
9090
fi
9191
cat matrix.json
9292
echo "matrix=$(jq -c . matrix.json)" >> "$GITHUB_OUTPUT"
@@ -146,6 +146,7 @@ jobs:
146146
--build-arg "MEGATRON_VERSION=${{ inputs.megatron_version }}" \
147147
--build-arg "FLAGOS_PYPI=${{ matrix.flagos_pypi }}" \
148148
--build-arg "EXTRA_PYPI=${{ matrix.extra_pypi }}" \
149+
--build-arg "APP_ENV=${{ matrix.app_env }}" \
149150
-t "${app_tag}" \
150151
-f app/megatron/Containerfile .
151152
echo "Built: ${app_tag}"

app/megatron/Containerfile

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,16 @@ RUN pip install \
5959
--extra-index-url "${EXTRA_PYPI}" \
6060
"megatron-core==${MEGATRON_VERSION}"
6161

62+
# --- App env ---------------------------------------------------
63+
64+
# Per-backend app env vars (configs.yaml env.app.{app}): baked into
65+
# /etc/profile.d/app_env.sh — sourced by the runtime's bash plumbing
66+
# (BASH_ENV=/etc/bash_env.sh sources /etc/profile.d/*.sh).
67+
ARG APP_ENV=""
68+
RUN if [ -n "${APP_ENV}" ]; then \
69+
printf '%s\n' "${APP_ENV}" > /etc/profile.d/app_env.sh; \
70+
fi
71+
6272
# --- Runtime --------------------------------------------------
6373

6474
WORKDIR /workspace

app/vllm/Containerfile

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ FROM ${RUNTIME_IMAGE}
3232

3333
# --- Build arguments ------------------------------------------
3434

35-
ARG VLLM_VERSION=0.20.2
35+
ARG VLLM_VERSION=0.24.0
3636

3737
# Repacked vllm wheel lives here — searched first.
3838
ARG FLAGOS_PYPI=""
@@ -50,10 +50,16 @@ ARG VLLM_VENDOR=cuda
5050

5151
# --- Install vllm (repacked wheel) ----------------------------
5252

53+
# The vendor PyPI carries the repacked wheel as vllm-{version}+flagos. The
54+
# +flagos pin is mandatory: a bare "vllm=={version}" would fall through to
55+
# the official wheel on the fallback index, whose torch>=2.11.0 requirement
56+
# would pull torch 2.11 over the runtime's pinned vendor torch (the exact
57+
# matrix disturbance repack exists to prevent — see
58+
# packaging/vllm/docs/report-vllm-0.24.0.md §1.2/§4.1).
5359
RUN pip install \
5460
--index-url "${FLAGOS_PYPI}" \
5561
--extra-index-url "${EXTRA_PYPI}" \
56-
"vllm==${VLLM_VERSION}"
62+
"vllm==${VLLM_VERSION}+flagos"
5763

5864
# --- Install vllm-plugin-FL (source) --------------------------
5965

@@ -69,6 +75,16 @@ RUN if [ -n "${PLUGIN_FL_REF}" ]; then \
6975
&& rm -rf /workspace/vllm-plugin-FL ; \
7076
fi
7177

78+
# --- App env ---------------------------------------------------
79+
80+
# Per-backend app env vars (configs.yaml env.app.{app}): baked into
81+
# /etc/profile.d/app_env.sh — sourced by the runtime's bash plumbing
82+
# (BASH_ENV=/etc/bash_env.sh sources /etc/profile.d/*.sh).
83+
ARG APP_ENV=""
84+
RUN if [ -n "${APP_ENV}" ]; then \
85+
printf '%s\n' "${APP_ENV}" > /etc/profile.d/app_env.sh; \
86+
fi
87+
7288
# --- Runtime --------------------------------------------------
7389

7490
ENV VLLM_PLUGINS=fl

configs.yaml

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,8 +51,11 @@
5151
# "env:" — environment split by which image consumes it:
5252
# base: vars baked into flagos-base-{vendor}-{backend}
5353
# runtime: vars baked into flagos-runtime-{vendor}-{backend}
54-
# base/runtime must mirror EXACTLY what the image sets — the
55-
# source of truth for per-image env documentation.
54+
# app: per-app vars baked into
55+
# flagos-app/{app}-{vendor}-{backend}, keyed by
56+
# app name (vllm / megatron)
57+
# base/runtime/app must mirror EXACTLY what the image sets —
58+
# the source of truth for per-image env documentation.
5659
# "deps:" — Python dependencies (torch stack, etc.).
5760
# "hardware:" — chip models the image targets (rendered as a prerequisite).
5861
# "driver:" — host driver version installed on the node — a prerequisite,
@@ -110,6 +113,12 @@ vendors:
110113
base:
111114
PATH: /usr/local/cuda/bin:$PATH
112115
LD_LIBRARY_PATH: /usr/local/cuda/lib64:$LD_LIBRARY_PATH
116+
app:
117+
vllm:
118+
# vLLM 0.24.0 unconditionally imports flashinfer (sampler switch
119+
# defaults to True). The runtime does not ship flashinfer — turn
120+
# the sampler off at the app layer.
121+
VLLM_USE_FLASHINFER_SAMPLER: "0"
113122
deps:
114123
- torch==2.10.0+cu128
115124
- torchaudio==2.10.0+cu128
@@ -130,6 +139,12 @@ vendors:
130139
base:
131140
PATH: /usr/local/cuda/bin:$PATH
132141
LD_LIBRARY_PATH: /usr/local/cuda/lib64:$LD_LIBRARY_PATH
142+
app:
143+
vllm:
144+
# vLLM 0.24.0 unconditionally imports flashinfer (sampler switch
145+
# defaults to True). The runtime does not ship flashinfer — turn
146+
# the sampler off at the app layer.
147+
VLLM_USE_FLASHINFER_SAMPLER: "0"
133148
deps:
134149
- torch==2.11.0+cu130
135150
- torchaudio==2.11.0+cu130

docs/gen_data.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -302,6 +302,11 @@ def image(prefix, kind, name, tag):
302302
"packages": runtime_packages(spec, configs.get("flaggems", "")),
303303
"env": env.get("runtime") or {},
304304
},
305+
"app": {
306+
# Per-app env vars (configs.yaml env.app.{app}) — consumed
307+
# by generate_matrix.py --app for app image builds.
308+
"env": env.get("app") or {},
309+
},
305310
}
306311
)
307312

packaging/vllm/docs/report-vllm-0.24.0.md

Lines changed: 104 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,9 @@
99
> 新的 Wheel 包在安装之后使用各 GPU/NPU 厂家所提供的运行时。
1010
> 为确保最终生成的软件堆栈有效,需要逐个后端(Backend)地到对应的物理环境执行验证,
1111
> 确保新的 Wheel 包及其依赖项能够正确安装、vLLM 软件栈可正常启动并执行推理任务。
12-
> 本文记录 **metax(沐曦)** 后端的验证过程与结果。
12+
> 本文记录 **metax(沐曦)****nvidia(英伟达)** 后端的验证过程与结果:
13+
> - metax 见 §3–§5(MACA SDK 3.7.2.1 / 3.8.1.3 × 双编译器,4 环境全通过);
14+
> - nvidia 见 §6(CUDA 12.8 / 13.3 × 双编译器,空模式,全通过)。
1315
>
1416
> 类似的工作也在 0.20.2 版本的 vLLM 上开展,相关记录见
1517
> [`report-vllm-0.20.2.md`](report-vllm-0.20.2.md)
@@ -18,13 +20,17 @@
1820

1921
## 0. 结论摘要(TL;DR)
2022

21-
结论:Metax 全线 4 种环境全部验证通过。
23+
结论:Metax 全线 4 种环境全部验证通过;NVIDIA 2 种 CUDA 环境(×双编译器)全部验证通过
2224

2325
- 构建 + 重新打包(wheel):✅ 通过(2026-08-12)
24-
- SDK(3.7.2.1)× FlagTree:✅ 通过(2026-08-14)
25-
- SDK(3.7.2.1)× Triton:✅ 通过(2026-08-13)
26-
- SDK(3.8.1.3)× FlagTree:✅ 通过(2026-08-15)
27-
- SDK(3.8.1.3)× Triton:✅ 通过(2026-08-15)
26+
- MACA(3.7.2.1)× FlagTree:✅ 通过(2026-08-14)
27+
- MACA(3.7.2.1)× Triton:✅ 通过(2026-08-13)
28+
- MACA(3.8.1.3)× FlagTree:✅ 通过(2026-08-15)
29+
- MACA(3.8.1.3)× Triton:✅ 通过(2026-08-15)
30+
- CUDA 12.8 × FlagTree:✅ 通过(2026-08-16,空模式)
31+
- CUDA 12.8 × Triton:✅ 通过(2026-08-16,空模式)
32+
- CUDA 13.3 × FlagTree:✅ 通过(2026-08-16,空模式)
33+
- CUDA 13.3 × Triton:✅ 通过(2026-08-16,空模式)
2834

2935
"通过" 意味着:1) vLLM 服务可以正常启动;2)使用 Qwen3-4B 模型可以执行正常推理服务;
3036

@@ -186,7 +192,7 @@ vLLM 顶层的依赖声明相应改成 `==X.Y.Z+flagos`,单步安装时命中
186192
修复:改用 `from flag_gems import reshape_and_cache_flash`
187193
(纯 Python 实现,签名逐参吻合)。
188194
**同一问题也存在于 0.20.2 版本适配中,已向插件提交 PR #333**
189-
另外,vllm-plugin-FL 的 0.3-dev 分支也不存在此修复(详 §6)。
195+
另外,vllm-plugin-FL 的 0.3-dev 分支也不存在此修复(详 §7)。
190196

191197
5. **Triton 3.0.0 编译器拒绝链式布尔操作**`A or B or C` 语法报
192198
"chained boolean operators not supported"。
@@ -251,7 +257,74 @@ vLLM 二次启动报 `ValueError: Free memory on device cuda:0 (2.02/63.59 GiB)
251257

252258
---
253259

254-
## 6. 版本推进协作问题
260+
## 6. NVIDIA(CUDA 12.8 / 13.3)详细记录
261+
262+
日期:2026-08-16
263+
节点:`h20`(H20 GPU,x86_64)
264+
镜像:`flagos-runtime-nvidia-cuda12.8:2.1.2` / `flagos-runtime-nvidia-cuda13.3:2.1.2`
265+
模型:Qwen3-4B(`/models/Qwen3-4B`,由 `/data/tqm/models` 挂载)
266+
参数:`--enforce-eager --dtype bfloat16`,端口 8031/8032
267+
268+
### 6.0 插件基线:v0.3.0-dev
269+
270+
NVIDIA 路径使用 vllm-plugin-FL 的 **`v0.3.0-dev` 分支**(官方 0.24.0 适配线,
271+
tar.gz 解包到 `/app/vllm-plugin-FL`;与 main 分支的差异与合并路线见 §7),
272+
在 dev 分支上 **无需任何 monkey-patch**
273+
274+
### 6.1 关键阻塞点与解决
275+
276+
1. **CUDA 平台无条件 import flashinfer**:0.24.0 的 CUDA 平台代码在
277+
`flashinfer_sampler_supported()` 中检查环境变量 `VLLM_USE_FLASHINFER_SAMPLER`
278+
(默认 True)后就 import flashinfer。Runtime 镜像未装 flashinfer,
279+
启动即报 import 错误。
280+
281+
解决:启动时设置 **`VLLM_USE_FLASHINFER_SAMPLER=0`**(环境变量开关,
282+
非代码修改)。日志确认:`FlashInfer top-p/top-k sampling disabled via
283+
VLLM_USE_FLASHINFER_SAMPLER=0`。
284+
285+
2. **插件安装必须 `--no-build-isolation`**:pip 构建隔离会独立下载
286+
pyproject 声明的构建依赖 —— 其中 `torch>=2.7.1` 从 pypi.org 拉取约 2.4GB,
287+
且会**用下载的 torch 构建插件**。这与 repack 的初衷(保护 Runtime 镜像中
288+
精心匹配的版本矩阵)直接冲突,绝不允许。
289+
290+
解决:先盘点 Runtime 环境已有工具链(setuptools 81.0.0、pybind11 3.0.3、
291+
ninja 1.13.0 已具备;缺 `wheel``scikit-build-core==0.11``cmake`),
292+
**厂商 PyPI 索引**`flagos-pypi-nvidia`)补齐缺失项,再
293+
`pip install -e . --no-build-isolation`(约 30 秒完成)。
294+
295+
长期方案:随 §7 合并路线将插件以 **wheel 形式发布**后,源安装路径整体消失
296+
—— 预编译 wheel 不需要任何构建工具链,无需把 `wheel`/`scikit-build-core`/
297+
`cmake` 常驻进共享 Runtime 镜像(它们只为单步源安装临时补齐)。
298+
299+
### 6.2 验证结果
300+
301+
**CUDA 12.8(torch 2.10.0+cu128,python 3.12)**
302+
303+
- flagtree 3.6.0(`/opt/flagtree`,默认):✅ 启动 + 推理通过
304+
- triton 3.6.0(`/opt/triton`):✅ 启动 + 推理通过
305+
- 安装:`pip install vllm==0.24.0+flagos` 单步(`Using cached
306+
vllm-0.24.0%2Bflagos-cp312-cp312-linux_x86_64.whl (7.8 MB)`,无新下载)
307+
308+
**CUDA 13.3(torch 2.11.0+cu130,python 3.12)**
309+
310+
- flagtree 3.6.0:✅ 启动 + 推理通过
311+
- triton 3.6.0:✅ 启动 + 推理通过
312+
- 安装:插件 `--no-build-isolation` + vllm 单步
313+
314+
两种编译器、两个 CUDA 版本的推理输出完全一致:
315+
`' Paris. The capital of Germany is Berlin. The capital of Italy is Rome.'`
316+
(finish=length,模型指纹 `vllm-0.24.0-423da8ca`)。
317+
318+
### 6.3 跨 CUDA 版本复用(重要结论)
319+
320+
12.8 与 13.3 同为 python 3.12,**共用一个 cp312 empty wheel**
321+
`vllm-0.24.0+flagos-cp312-cp312-linux_x86_64.whl`)。13.3 验证同时回答了
322+
"相同 cp 版本的 Wheel 是否可跨 CUDA 使用":**12.8 构建的 wheel 直接在 13.3
323+
(cu130)上单步安装并运行通过**。是否可跨 OS/架构(如 aarch64)仍待验证。
324+
325+
---
326+
327+
## 7. 版本推进协作问题
255328

256329
vllm-plugin-FL 项目组在 **`v0.3.0-dev`** 分支上展开 vLLM 0.24.0 适配工作。
257330
#252 处与 main 分叉,尚未合入 main。适配主线:
@@ -292,6 +365,9 @@ build-infra 验证用的基线是 main + PR #377,vllm-plugin-FL 的正式适
292365
**待定事项:**
293366
- [ ] **确认 0.24.0 正式发布线**:main(+ 我们补丁)还是 v0.3.0-dev?
294367
如果以 dev 为基线,§4 的验证要在 dev 分支上重验。
368+
**§6 的 NVIDIA 验证已在 dev 基线上完成(空模式、零 patch、双编译器全过)**
369+
证明 dev 分支在 CUDA 上可直接交付;metax 侧则需要确认 3.0.0 编译器问题
370+
的 4 个补丁是否 upstream。
295371
- [ ] `_patch_torch_accelerator``accelerator_compat.py` 去重:
296372
保留哪个版本;是否把 reset 的 try/except 兜底 backport 到 dev。
297373
- [ ] 确认 dev 分支在 triton 3.0.0 路径是否缺问题 3/5/6 的修复(serve 冒烟即知)。
@@ -301,21 +377,35 @@ build-infra 验证用的基线是 main + PR #377,vllm-plugin-FL 的正式适
301377

302378
---
303379

304-
## 7. 遗留事项
380+
## 8. 遗留事项
305381

306382
- [ ] `setuptools 84.0.0` 不满足 pyproject 中 `<81` 要求 —— 非致命问题,先不动,留意。
307-
- [ ] 0.24.0 其余后端(nvidia、mthreads、hygon、iluvatar、enflame、sunrise 等)的验证
383+
- [ ] 0.24.0 其余后端(mthreads、hygon、iluvatar、enflame、sunrise、cambricon、
384+
ascend、kunlunxin 等)的验证
308385

309386
---
310387

311388
## 附录 · 验证命令(容器内)
312389

390+
metax 形式(含 `compiler` 切换 + `VLLM_USE_FLASHINFER_SAMPLER=0`,NVIDIA 通用):
391+
313392
```bash
314-
cd /app/vllm-plugin-FL && PYTHONPATH=/opt/triton:/opt/flagtree \
393+
cd /app/vllm-plugin-FL && compiler flagtree && VLLM_USE_FLASHINFER_SAMPLER=0 \
315394
nohup /flagos/bin/python -m vllm.entrypoints.openai.api_server \
316-
--model /data/models/Qwen/Qwen3-4B --port 8031 --enforce-eager --dtype bfloat16 \
317-
> /tmp/serve-0.24.0.log 2>&1 &
395+
--model /models/Qwen3-4B --port 8031 --enforce-eager --dtype bfloat16 \
396+
> /tmp/serve.log 2>&1 &
318397

319398
curl -s localhost:8031/v1/completions -H 'Content-Type: application/json' \
320-
-d '{"model":"/data/models/Qwen/Qwen3-4B","prompt":"The capital of France is","max_tokens":16,"temperature":0}'
399+
-d '{"model":"/models/Qwen3-4B","prompt":"The capital of France is","max_tokens":16,"temperature":0}'
400+
```
401+
402+
NVIDIA 插件安装 —— 仅当前源安装路径需要(必须 `--no-build-isolation`,缺失工具链
403+
从厂商 PyPI 补齐);插件以 wheel 形式发布后(§7 合并路线),此块整体省略:
404+
405+
```bash
406+
/flagos/bin/pip install --no-cache-dir wheel scikit-build-core==0.11 cmake \
407+
-i https://resource.flagos.net/repository/flagos-pypi-nvidia/simple/ \
408+
--extra-index-url https://mirrors.aliyun.com/pypi/simple
409+
cd /app/vllm-plugin-FL && /flagos/bin/pip install -e . --no-build-isolation
410+
/flagos/bin/pip install vllm==0.24.0+flagos
321411
```

packaging/vllm/docs/vllm-verification-matrix.md

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,8 @@
2222

2323
| 厂商 | 后端 | 0.20.2(T) | 0.20.2(F) | 0.24.0(T) | 0.24.0(F) |
2424
|---|---|---|---|---|---|
25-
| 英伟达 | CUDA 12.8 ||| | |
26-
| 英伟达 | CUDA 13.3 ||| | |
25+
| 英伟达 | CUDA 12.8 ||| | |
26+
| 英伟达 | CUDA 13.3 ||| | |
2727
| 昇腾 | CANN 8.5.0 |||||
2828
| 昇腾 | CANN 9.0.0 |||||
2929
| 寒武纪 | NEUWARE 4.4.3 |||||
@@ -85,8 +85,18 @@
8585
triton 3.0.0(XTDK LLVM19 空 SetVector 断言)。应用层无法绕过,已交编译器团队,
8686
详见 [kunlunxin-xpu-triton-attention-compiler-bug.md](kunlunxin-xpu-triton-attention-compiler-bug.md)
8787

88-
**0.24.0(截至 2026-08-15)**
89-
88+
**0.24.0(截至 2026-08-16)**
89+
90+
- **nvidia-cuda12.8** ✅(2026-08-16,空模式双编译器):flagtree 3.6.0 ✅、
91+
triton 3.6.0 ✅(`/opt/triton`),均通过 Qwen3-4B E2E 验证,指纹 `vllm-0.24.0-423da8ca`
92+
- **nvidia-cuda13.3** ✅(2026-08-16,空模式双编译器):flagtree 3.6.0 ✅、
93+
triton 3.6.0 ✅,均通过 Qwen3-4B E2E 验证,指纹 `vllm-0.24.0-423da8ca`
94+
同时验证了 **cp312 empty wheel 跨 CUDA 复用**:12.8(torch 2.10.0+cu128)构建的
95+
同一 Wheel 直接在 13.3(torch 2.11.0+cu130)上安装运行。
96+
- **NVIDIA 路径统一要点**(详见 `report-vllm-0.24.0.md` §6):
97+
插件基线 **v0.3.0-dev**;CUDA 平台无条件 import flashinfer,
98+
需环境变量 `VLLM_USE_FLASHINFER_SAMPLER=0` 关闭采样器;
99+
插件安装必须 `--no-build-isolation`(构建隔离会从 pypi.org 下载 torch≈2.4GB)。
90100
- **metax 四环境全 ✅**(2026-08-13~15):MACA 3.7.2.1
91101
(triton 3.0.0 / flagtree 0.6.1+metax3.6)× 两编译器、MACA 3.8.1.3
92102
(triton 3.6.0 / flagtree 0.6.1+metax3.6)× 两编译器。
@@ -95,13 +105,16 @@
95105
- **triton 3.0.0 需 4 个 monkey-patch**`vllm024_compat.py`,老 SDK 特有):
96106
`_load_ptr` constexpr 解包、`_penalties_kernel` 链式布尔加括号、
97107
`get_top_k_top_p``pool` 的 UVA CPU 索引 + 移回设备;新 SDK(triton 3.6.0)无需。
98-
- **0.24.0 其余后端待验证**nvidia, mthreads, hygon, iluvatar, enflame, sunrise,
108+
- **0.24.0 其余后端待验证**:mthreads, hygon, iluvatar, enflame, sunrise,
99109
cambricon, ascend, kunlunxin 等。
100110

101111
**跨版本事实**
102112

103113
- 0.24.0 empty wheel 绑定 `cp312-cp312-linux_x86_64`
104114
0.20.2 是 `py3-none-any`(纯 Python,可跨 cp 复用)。
115+
- **cp312 wheel 跨 CUDA 版本可复用**(实测):cuda12.8 构建的 `vllm-0.24.0+flagos`
116+
wheel 在 cuda13.3(torch 2.11.0+cu130)上单步安装运行 ✅。
117+
是否可跨 OS/架构(如 aarch64)仍待验证。
105118
- 所有适配补丁收敛在 vllm-plugin-FL 插件侧,不修改官方 vLLM。
106119

107120
## 已知问题 / 阻塞

0 commit comments

Comments
 (0)