Skip to content

Commit 4edf977

Browse files
authored
[doc] feat: update Qwen3.5 NPU guide (verl-project#6667)
### What does this PR do? update qwen3.5 ascend doc for 35b and 122b size ### Checklist Before Starting - [ ] Search for similar PRs. Paste at least one query link here: ... - [x] Format the PR title as `[{modules}] {type}: {description}` (This will be checked by the CI) - `{modules}` include `fsdp`, `megatron`, `veomni`, `sglang`, `vllm`, `rollout`, `trainer`, `ci`, `training_utils`, `recipe`, `hardware`, `deployment`, `ray`, `worker`, `single_controller`, `misc`, `perf`, `model`, `algo`, `env`, `tool`, `ckpt`, `doc`, `data`, `cfg`, `reward`, `fully_async`, `one_step_off` - If this PR involves multiple modules, separate them with `,` like `[megatron, fsdp, doc]` - `{type}` is in `feat`, `fix`, `refactor`, `chore`, `test` - If this PR breaks any API (CLI arguments, config, function signature, etc.), add `[BREAKING]` to the beginning of the title. - Example: `[BREAKING][fsdp, megatron] feat: dynamic batching` ### Test > For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluation results, etc. ### API and Usage Example > Demonstrate how the API changes if any, and provide usage example(s) if possible. ```python # Add code snippet or script demonstrating how to use this ``` ### Design & Code Changes > Demonstrate the high-level design if this PR is complex, and list the specific changes. ### Checklist Before Submitting > [!IMPORTANT] > Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review. - [x] Read the [Contribute Guide](https://github.com/verl-project/verl/blob/main/CONTRIBUTING.md). - [x] Apply [pre-commit checks](https://github.com/verl-project/verl/blob/main/CONTRIBUTING.md#code-linting-and-formatting): `pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always` - [ ] Add / Update [the documentation](https://github.com/verl-project/verl/tree/main/docs). - [ ] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/verl-project/verl/tree/main/.github/workflows) to cover all the code. If not feasible, explain why: ... - [ ] Once your PR is ready for CI, send a message in [the `ci-request` channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the `verl` Slack workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ). (If not accessible, please try [the Feishu group (飞书群)](https://applink.larkoffice.com/client/chat/chatter/add_by_link?link_token=772jd4f1-cd91-441e-a820-498c6614126a).) - [ ] If your PR is related to the `recipe` submodule, please also update the reference to the submodule commit via `git submodule update --remote` or `cd recipe && git pull origin main`.
1 parent 5a38699 commit 4edf977

4 files changed

Lines changed: 137 additions & 91 deletions

File tree

docs/ascend_tutorial/index.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ Last updated: 06/05/2026.
2929
model_support/examples/ascend_vllm_best_practices
3030
model_support/examples/dapo_multi_model_optimization_practice
3131
model_support/examples/gspo_optimization_practice
32-
model_support/examples/qwen3_5_122b_npu
32+
model_support/examples/qwen3_5_megatron_npu
3333

3434
.. toctree::
3535
:maxdepth: 1

docs/ascend_tutorial/model_support/examples/qwen3_5_122b_npu.md

Lines changed: 0 additions & 89 deletions
This file was deleted.
Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
# Qwen3.5 Megatron NPU 使用指南
2+
3+
Last updated: 06/08/2026.
4+
5+
本文用于指导在 Ascend NPU 上使用 verl + Megatron + vLLM 跑通 Qwen3.5-35B-A3B 和 Qwen3.5-122B-A10B GRPO 示例。
6+
7+
## 版本要求
8+
9+
| software | version |
10+
| --- | --- |
11+
| Docker image | `quay.io/ascend/verl:verl-9.0.0-a3-ubuntu22.04-py3.11-latest` |
12+
| verl | commit `cdd9014f` |
13+
| Python | 3.11 |
14+
| CANN | 9.0.0 |
15+
| Megatron-LM | 0.16.1 |
16+
| MindSpeed | 0.16.0 |
17+
| Megatron-Bridge | `de93536e` |
18+
19+
建议直接使用上表中的镜像:
20+
21+
```bash
22+
docker pull quay.io/ascend/verl:verl-9.0.0-a3-ubuntu22.04-py3.11-latest
23+
```
24+
25+
启动容器后,将 verl 固定到指定 commit:
26+
27+
```bash
28+
git checkout cdd9014f
29+
```
30+
31+
脚本注释中列出的额外依赖如下;镜像已包含的依赖不需要重复安装:
32+
33+
```bash
34+
pip install viztracer flash-linear-attention nvidia-modelopt nvidia-ml-py nvidia-resiliency-ext megatron-energon
35+
```
36+
37+
## 模型和脚本
38+
39+
| model | HF model | script |
40+
| --- | --- | --- |
41+
| Qwen3.5-35B-A3B | `Qwen/Qwen3.5-35B-A3B` | `examples/grpo_trainer/run_qwen3_5_35b_megatron.sh` |
42+
| Qwen3.5-122B-A10B | `Qwen/Qwen3.5-122B-A10B` | `examples/grpo_trainer/run_qwen3_5_122b_a10b_megatron.sh` |
43+
44+
## 硬件和并行配置
45+
46+
示例脚本默认使用如下 NPU 配置,可以通过同名环境变量覆盖:
47+
48+
| model | nnodes | devices per node | TP | PP | CP | EP | ETP | GEN_TP |
49+
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
50+
| Qwen3.5-35B-A3B | 1 | 16 | 2 | 2 | 1 | 8 | 1 | 8 |
51+
| Qwen3.5-122B-A10B | 4 | 16 | 2 | 4 | 1 | 16 | 1 | 16 |
52+
53+
## 数据和模型准备
54+
55+
脚本默认使用 Geo3K 数据集,并会下载到 `$HOME/data/geo3k`
56+
57+
```bash
58+
hf download tyzhu/geo3k --repo-type dataset --local-dir $HOME/data/geo3k
59+
```
60+
61+
模型权重可以使用 Hugging Face 模型名,也可以提前下载到本地路径:
62+
63+
```bash
64+
hf download Qwen/Qwen3.5-35B-A3B --local-dir /path/to/Qwen3.5-35B-A3B
65+
hf download Qwen/Qwen3.5-122B-A10B --local-dir /path/to/Qwen3.5-122B-A10B
66+
```
67+
68+
## 启动训练
69+
70+
训练前需要先启动 Ray 集群。通用多节点说明可参考 [Multinode Training](../../../start/multinode.rst),Ascend 多节点脚本示例可参考 [Ascend SGLang Best Practices](ascend_sglang_best_practices.rst)
71+
72+
最小启动方式如下,单机任务只需要执行 head 节点命令;多机任务需要在其他节点执行 worker 节点命令。`MASTER_ADDR` 在所有节点上保持一致,`CURRENT_IP` 设置为当前节点 IP。
73+
74+
```bash
75+
MASTER_ADDR=<head-node-ip>
76+
CURRENT_IP=<current-node-ip>
77+
NPUS_PER_NODE=16
78+
79+
# head node
80+
ray start --head --port 6766 --dashboard-host=$MASTER_ADDR --node-ip-address=$CURRENT_IP --dashboard-port=8260 --resources='{"NPU": '$NPUS_PER_NODE'}'
81+
82+
# worker nodes, only needed for multi-node jobs
83+
ray start --address="$MASTER_ADDR:6766" --node-ip-address=$CURRENT_IP --resources='{"NPU": '$NPUS_PER_NODE'}'
84+
85+
ray status
86+
```
87+
88+
通过 `ray status` 确认 NPU 资源数量符合预期后,在主节点执行训练脚本。Qwen3.5-35B-A3B 默认需要 16 个 NPU 资源,Qwen3.5-122B-A10B 默认需要 64 个 NPU 资源。
89+
90+
### Qwen3.5-35B-A3B
91+
92+
```bash
93+
export DEVICE=npu
94+
export HF_MODEL_PATH=/path/to/Qwen3.5-35B-A3B
95+
96+
bash examples/grpo_trainer/run_qwen3_5_35b_megatron.sh
97+
```
98+
99+
如果需要覆盖数据路径:
100+
101+
```bash
102+
DEVICE=npu \
103+
HF_MODEL_PATH=/path/to/Qwen3.5-35B-A3B \
104+
train_path=/path/to/train.parquet \
105+
test_path=/path/to/test.parquet \
106+
bash examples/grpo_trainer/run_qwen3_5_35b_megatron.sh
107+
```
108+
109+
### Qwen3.5-122B-A10B
110+
111+
```bash
112+
export DEVICE=npu
113+
export HF_MODEL_PATH=/path/to/Qwen3.5-122B-A10B
114+
115+
bash examples/grpo_trainer/run_qwen3_5_122b_a10b_megatron.sh
116+
```
117+
118+
如果需要覆盖数据、保存路径或并行配置:
119+
120+
```bash
121+
DEVICE=npu \
122+
HF_MODEL_PATH=/path/to/Qwen3.5-122B-A10B \
123+
train_files=/path/to/train.parquet \
124+
test_files=/path/to/test.parquet \
125+
save_path=/path/to/checkpoints \
126+
n_devices_per_node=16 \
127+
nnodes=4 \
128+
bash examples/grpo_trainer/run_qwen3_5_122b_a10b_megatron.sh
129+
```
130+
131+
## 注意事项
132+
133+
- 脚本会通过 `torch_npu` 自动识别 NPU 环境;如需手动指定,设置 `DEVICE=npu`
134+
- Qwen3.5 的 Gated Delta Net 当前不使用 packed sequence,因此脚本中保持 `use_remove_padding=False``use_dynamic_bsz=False`
135+
- NPU 分支会设置 `vanilla_mbridge=False``use_flash_attn=True``moe_token_dispatcher_type=alltoall` 等 Ascend 适配参数。

docs/ascend_tutorial/model_support/model_and_algorithm_support.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ Last updated: 05/14/2026.
4646
| 38 | Qwen3.5-27B | GRPO | [27B](https://huggingface.co/Qwen/Qwen3.5-27B) | FSDP(MindSpeed-MM) | vllm | [`link`](https://github.com/verl-project/verl-ascend-recipe/tree/main/grpo_mindspeed_mm/run_qwen3_5-27b_npu.sh) | - | Atlas 200T A2 Box16, Atlas 800T A3 |
4747
| 39 | Qwen3.5-35B-A3B | GRPO | [35B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B) | FSDP2 | vllm | [`link`](https://github.com/verl-project/verl/blob/main/examples/grpo_trainer/run_qwen3_5_35b_fsdp.sh) | - | Atlas 200T A2 Box16, Atlas 800T A3 |
4848
| 40 | Qwen3.5-35B-A3B | GRPO | [35B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B) | FSDP2(MindSpeed-MM) | vllm | [`link`](https://github.com/verl-project/verl-ascend-recipe/tree/main/grpo_mindspeed_mm/run_qwen3_5-35b_npu.sh) | - | Atlas 200T A2 Box16, Atlas 800T A3 |
49-
| 41 | Qwen3.5-122B-A10B | GRPO | [122B](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) | Megatron | vllm | [`link`](https://github.com/verl-project/verl/blob/main/examples/grpo_trainer/run_qwen3_5_122b_a10b_megatron.sh) | [`link`](examples/qwen3_5_122b_npu.md) | Atlas 800T A3 |
49+
| 41 | Qwen3.5-122B-A10B | GRPO | [122B](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) | Megatron | vllm | [`link`](https://github.com/verl-project/verl/blob/main/examples/grpo_trainer/run_qwen3_5_122b_a10b_megatron.sh) | [`link`](examples/qwen3_5_megatron_npu.md) | Atlas 800T A3 |
5050
| 42 | Qwen3-Next-80B-A3B-Instruct | GRPO | [80B](https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct) | FSDP2 | vllm | [`link`](https://github.com/verl-project/verl/blob/main/examples/grpo_trainer/run_qwen3_next_80b_a3b_fsdp.sh) | - | Atlas 900 A2 PODc, Atlas 200T A2 Box16, Atlas 800T A3 |
5151
| 43 | DeepSeek-V3 | GRPO | [671B](https://huggingface.co/deepseek-ai/DeepSeek-V3) | Megatron | vllm | [`link`](https://github.com/verl-project/verl-ascend-recipe/tree/main/r1_ascend/run_deepseekv3_671b_grpo_megatron_npu.sh) | - | Atlas 200T A2 Box16, Atlas 800T A3 |
5252
| 44 | Qwen3-1.7B | GRPO | [1.7B](https://huggingface.co/Qwen/Qwen3-1.7B) | VeOmni | vllm | [`link`](https://github.com/verl-project/verl-ascend-recipe/blob/main/verl_ascend_practice/run_qwen3_1_7b_npu.sh) | - | Atlas 800T A3 |

0 commit comments

Comments
 (0)