Skip to content

Commit e66f5ab

Browse files
cuichenxcuichenx
committed
merge(main): refresh GLM-4.5V assistant mask fix
Signed-off-by: Chen Cui <chcui@nvidia.com>
2 parents 5898845 + 16846a8 commit e66f5ab

94 files changed

Lines changed: 9583 additions & 1215 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/fern/versions/0.4.2/pages/models/vlm/nemotron-nano-v2-vl.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ We use the following environment variables throughout this page
2424

2525
Unless explicitly stated, any megatron model path in the commands below should NOT contain the iteration number
2626
`iter_xxxxxx`. For more details on checkpointing, please see
27-
[here](../../training/checkpointing.md#checkpoint-contents)
27+
[here](../../training/checkpointing.md#checkpoint-contents)
2828

2929
</Tip>
3030

‎docs/fern/versions/0.4.2/pages/models/vlm/qwen2.5-vl.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ We use the following environment variables throughout this page
1212

1313
Unless explicitly stated, any megatron model path in the commands below should NOT contain the iteration number
1414
`iter_xxxxxx`. For more details on checkpointing, please see
15-
[here](../../training/checkpointing.md#checkpoint-contents)
15+
[here](../../training/checkpointing.md#checkpoint-contents)
1616

1717
</Tip>
1818

‎docs/fern/versions/0.4.2/pages/models/vlm/qwen3-vl.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ We use the following environment variables throughout this page
1010
- `MEGATRON_MODEL_PATH=/models/Qwen3-VL-8B-Instruct` (feel free to set your own path)
1111
Unless explicitly stated, any megatron model path in the commands below should NOT contain the iteration number
1212
`iter_xxxxxx`. For more details on checkpointing, please see
13-
[here](../../training/checkpointing.md#checkpoint-contents)
13+
[here](../../training/checkpointing.md#checkpoint-contents)
1414

1515
</Tip>
1616

‎docs/fern/versions/nightly/pages/models/qwen/qwen.mdx‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -218,9 +218,9 @@ uv run python examples/conversion/hf_to_megatron_generate_text.py \
218218

219219
| Model | Mode | TP | PP | EP | Total GPUs | Use Case |
220220
|-------|------|----|----|----|-----------:|----------|
221-
| **Qwen3-30B-A3B** | Pretrain | 1 | 1 | 8 | 8 | Pre-training (single node) |
222-
| **Qwen3-30B-A3B** | Full SFT | 1 | 1 | 8 | 8 | Full supervised finetuning |
223-
| **Qwen3-30B-A3B** | LoRA/DoRA | 1 | 1 | 8 | 8 | PEFT finetuning (single node) |
221+
| **Qwen3-30B-A3B** | Pretrain | 1 | 1 | 16 | 16 | Pre-training (2 nodes) |
222+
| **Qwen3-30B-A3B** | Full SFT | 1 | 1 | 16 | 16 | Full supervised finetuning (2 nodes) |
223+
| **Qwen3-30B-A3B** | LoRA/DoRA | 4 | 1 | 4 | 4 | PEFT finetuning |
224224
| **Qwen3-235B-A22B** | Pretrain | 2 | 8 | 32 | 512 | Pre-training (64 nodes) |
225225
| **Qwen3-235B-A22B** | Full SFT | 2 | 8 | 32 | 512 | Full supervised finetuning (64 nodes) |
226226
| **Qwen3-235B-A22B** | LoRA/DoRA | 2 | 8 | 32 | 512 | PEFT finetuning (64 nodes) |
@@ -239,7 +239,7 @@ config = qwen3_30b_a3b_pretrain_config(
239239
train_iters=500_000,
240240
global_batch_size=2048,
241241
seq_length=4096,
242-
# Uses TP=1, PP=1, EP=8 (8 GPUs) automatically
242+
# Uses TP=1, PP=1, EP=16 (16 GPUs) automatically
243243
)
244244
```
245245

@@ -272,7 +272,7 @@ config = qwen3_30b_a3b_sft_config(
272272
train_iters=1000,
273273
global_batch_size=64,
274274
finetune_lr=5e-6,
275-
# Uses TP=1, PP=1, EP=8 (8 GPUs) automatically
275+
# Uses TP=1, PP=1, EP=16 (16 GPUs) automatically
276276
)
277277
```
278278

@@ -288,7 +288,7 @@ config = qwen3_30b_a3b_peft_config(
288288
train_iters=1000,
289289
global_batch_size=128,
290290
finetune_lr=1e-4,
291-
# Uses TP=1, PP=1, EP=8 (8 GPUs) automatically
291+
# Uses TP=4, PP=1, EP=4 (4 GPUs) automatically
292292
)
293293
```
294294

‎docs/fern/versions/nightly/pages/models/qwen/qwen3-vl.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ We use the following environment variables throughout this page
1010
- `MEGATRON_MODEL_PATH=/models/Qwen3-VL-8B-Instruct` (feel free to set your own path)
1111
Unless explicitly stated, any megatron model path in the commands below should NOT contain the iteration number
1212
`iter_xxxxxx`. For more details on checkpointing, please see
13-
[here](../../training/checkpointing.md#checkpoint-contents)
13+
[here](../../training/checkpointing.md#checkpoint-contents)
1414

1515
</Tip>
1616

‎docs/models/qwen/qwen.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -218,9 +218,9 @@ uv run python examples/conversion/hf_to_megatron_generate_text.py \
218218

219219
| Model | Mode | TP | PP | EP | Total GPUs | Use Case |
220220
|-------|------|----|----|----|-----------:|----------|
221-
| **Qwen3-30B-A3B** | Pretrain | 1 | 1 | 8 | 8 | Pre-training (single node) |
222-
| **Qwen3-30B-A3B** | Full SFT | 1 | 1 | 8 | 8 | Full supervised finetuning |
223-
| **Qwen3-30B-A3B** | LoRA/DoRA | 1 | 1 | 8 | 8 | PEFT finetuning (single node) |
221+
| **Qwen3-30B-A3B** | Pretrain | 1 | 1 | 16 | 16 | Pre-training (2 nodes) |
222+
| **Qwen3-30B-A3B** | Full SFT | 1 | 1 | 16 | 16 | Full supervised finetuning (2 nodes) |
223+
| **Qwen3-30B-A3B** | LoRA/DoRA | 4 | 1 | 4 | 4 | PEFT finetuning |
224224
| **Qwen3-235B-A22B** | Pretrain | 2 | 8 | 32 | 512 | Pre-training (64 nodes) |
225225
| **Qwen3-235B-A22B** | Full SFT | 2 | 8 | 32 | 512 | Full supervised finetuning (64 nodes) |
226226
| **Qwen3-235B-A22B** | LoRA/DoRA | 2 | 8 | 32 | 512 | PEFT finetuning (64 nodes) |
@@ -239,7 +239,7 @@ config = qwen3_30b_a3b_pretrain_config(
239239
train_iters=500_000,
240240
global_batch_size=2048,
241241
seq_length=4096,
242-
# Uses TP=1, PP=1, EP=8 (8 GPUs) automatically
242+
# Uses TP=1, PP=1, EP=16 (16 GPUs) automatically
243243
)
244244
```
245245

@@ -272,7 +272,7 @@ config = qwen3_30b_a3b_sft_config(
272272
train_iters=1000,
273273
global_batch_size=64,
274274
finetune_lr=5e-6,
275-
# Uses TP=1, PP=1, EP=8 (8 GPUs) automatically
275+
# Uses TP=1, PP=1, EP=16 (16 GPUs) automatically
276276
)
277277
```
278278

@@ -288,7 +288,7 @@ config = qwen3_30b_a3b_peft_config(
288288
train_iters=1000,
289289
global_batch_size=128,
290290
finetune_lr=1e-4,
291-
# Uses TP=1, PP=1, EP=8 (8 GPUs) automatically
291+
# Uses TP=4, PP=1, EP=4 (4 GPUs) automatically
292292
)
293293
```
294294

‎docs/models/qwen/qwen3-vl.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ We use the following environment variables throughout this page
1010
- `MEGATRON_MODEL_PATH=/models/Qwen3-VL-8B-Instruct` (feel free to set your own path)
1111
Unless explicitly stated, any megatron model path in the commands below should NOT contain the iteration number
1212
`iter_xxxxxx`. For more details on checkpointing, please see
13-
[here](../../training/checkpointing.md#checkpoint-contents)
13+
[here](../../training/checkpointing.md#checkpoint-contents)
1414
```
1515

1616
## Examples
@@ -20,4 +20,4 @@ For checkpoint conversion, inference, finetuning recipes, and step-by-step train
2020
## Hugging Face Model Cards
2121
- Qwen3-VL-8B: `https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct`
2222
- Qwen3-VL-30B-A3B (MoE): `https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Instruct`
23-
- Qwen3-VL-235B-A22B (MoE): `https://huggingface.co/Qwen/Qwen3-VL-235B-A22B-Instruct`
23+
- Qwen3-VL-235B-A22B (MoE): `https://huggingface.co/Qwen/Qwen3-VL-235B-A22B-Instruct`

‎docs/performance-summary-archive.md‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,107 @@ The performance data includes:
3939

4040
---
4141

42+
## 26.06 NeMo Container
43+
44+
### Pre-Training Performance
45+
46+
#### Model: LLAMA3.1_405B
47+
48+
| System | #-GPUs | Precision | GBS | MBS | Sequence Length | TP | PP | CP | VP | EP | Tokens / sec / GPU | Model TFLOP / sec / GPU |
49+
|--------|--------|-----------|-----|-----|-----------------|----|----|----|----|----|-----------------------|-------------------------|
50+
| DGX-GB300 | 256 | FP8 | 1536 | 1 | 8192 | 4 | 8 | 1 | 4 | n/a | 1048 | 2646 |
51+
| DGX-GB300 | 256 | MXFP8 | 1536 | 1 | 8192 | 2 | 8 | 2 | 4 | n/a | 952 | 2403 |
52+
| DGX-GB300 | 256 | NVFP4 | 1536 | 1 | 8192 | 4 | 8 | 1 | 4 | n/a | 1413 | 3575 |
53+
| DGX-GB200 | 256 | FP8 | 1536 | 1 | 8192 | 4 | 16 | 1 | 4 | n/a | 843 | 2129 |
54+
| DGX-GB200 | 256 | MXFP8 | 1536 | 1 | 8192 | 4 | 16 | 1 | 8 | n/a | 783 | 1976 |
55+
| DGX-GB200 | 256 | NVFP4 | 1536 | 1 | 8192 | 4 | 16 | 1 | 8 | n/a | 1166 | 2944 |
56+
| DGX-H100 | 1024 | FP8 | 1536 | 1 | 8192 | 8 | 8 | 2 | 8 | n/a | 326 | 822 |
57+
58+
#### Model: DeepSeekV3
59+
60+
| System | #-GPUs | Precision | GBS | MBS | Sequence Length | TP | PP | CP | VP | EP | Tokens / sec / GPU | Model TFLOP / sec / GPU |
61+
|--------|--------|-----------|-----|-----|-----------------|----|----|----|----|----|-----------------------|-------------------------|
62+
| DGX-GB300 | 256 | MXFP8 | 4096 | 1 | 4096 | 1 | 2 | 1 | 8 | 32 | 6338 | 1648 |
63+
| DGX-GB300 | 256 | MXFP8 | 15360 | 1 | 4096 | 1 | 2 | 1 | 8 | 32 | 6422 | 1670 |
64+
| DGX-GB200 | 256 | MXFP8 | 4096 | 1 | 4096 | 1 | 4 | 1 | 4 | 64 | 4969 | 1292 |
65+
| DGX-B300 | 256 | MXFP8 | 4096 | 2 | 4096 | 1 | 8 | 1 | n/a | 8 | 3541 | 920 |
66+
67+
#### Model: GPT OSS 120B
68+
69+
| System | #-GPUs | Precision | GBS | MBS | Sequence Length | TP | PP | CP | VP | EP | Tokens / sec / GPU | Model TFLOP / sec / GPU |
70+
|--------|--------|-----------|-----|-----|-----------------|----|----|----|----|----|-----------------------|-------------------------|
71+
| DGX-GB300 | 64 | MXFP8 | 1280 | 4 | 4096 | 1 | 1 | 1 | n/a | 16 | 33166 | 1081 |
72+
| DGX-GB200 | 64 | MXFP8 | 1280 | 4 | 4096 | 1 | 1 | 1 | n/a | 64 | 28947 | 943 |
73+
| DGX-B300 | 64 | MXFP8 | 1280 | 4 | 4096 | 1 | 1 | 1 | n/a | 8 | 18534 | 604 |
74+
75+
#### Model: Qwen3_30B_a3B
76+
77+
| System | #-GPUs | Precision | GBS | MBS | Sequence Length | TP | PP | CP | VP | EP | Tokens / sec / GPU | Model TFLOP / sec / GPU |
78+
|--------|--------|-----------|-----|-----|-----------------|----|----|----|----|----|-----------------------|-------------------------|
79+
| DGX-GB300 | 8 | MXFP8 | 512 | 8 | 4096 | 1 | 1 | 1 | n/a | 8 | 45275 | 1041 |
80+
| DGX-GB200 | 8 | MXFP8 | 512 | 4 | 4096 | 1 | 1 | 1 | n/a | 8 | 40706 | 936 |
81+
| DGX-B300 | 8 | MXFP8 | 512 | 8 | 4096 | 1 | 1 | 1 | n/a | 8 | 40769 | 938 |
82+
| DGX-H100 | 16 | FP8 | 1024 | 1 | 4096 | 1 | 1 | 1 | n/a | 16 | 8826 | 203 |
83+
84+
#### Model: Qwen3_235B_a22B
85+
86+
| System | #-GPUs | Precision | GBS | MBS | Sequence Length | TP | PP | CP | VP | EP | Tokens / sec / GPU | Model TFLOP / sec / GPU |
87+
|--------|--------|-----------|-----|-----|-----------------|----|----|----|----|----|-----------------------|-------------------------|
88+
| DGX-GB300 | 256 | MXFP8 | 8192 | 2 | 4096 | 1 | 4 | 1 | 12 | 32 | 9015 | 1335 |
89+
| DGX-GB200 | 256 | MXFP8 | 8192 | 1 | 4096 | 1 | 8 | 1 | 3 | 32 | 7376 | 1092 |
90+
91+
#### Model: Kimi_K2
92+
93+
| System | #-GPUs | Precision | GBS | MBS | Sequence Length | TP | PP | CP | VP | EP | Tokens / sec / GPU | Model TFLOP / sec / GPU |
94+
|--------|--------|-----------|-----|-----|-----------------|----|----|----|----|----|-----------------------|-------------------------|
95+
| DGX-GB300 | 256 | MXFP8 | 4096 | 2 | 4096 | 1 | 4 | 1 | 4 | 64 | 5372 | 1099 |
96+
97+
- Muon optimizer was used for pre-training Kimi-K2.
98+
99+
#### Model: Nemotron_3_Nano
100+
101+
| System | #-GPUs | Precision | GBS | MBS | Sequence Length | TP | PP | CP | VP | EP | Tokens / sec / GPU | Model TFLOP / sec / GPU |
102+
|--------|--------|-----------|-----|-----|-----------------|----|----|----|----|----|-----------------------|-------------------------|
103+
| DGX-GB300 | 8 | MXFP8 | 512 | 4 | 8192 | 1 | 1 | 1 | n/a | 8 | 39749 | 885 |
104+
| DGX-GB200 | 8 | MXFP8 | 512 | 2 | 8192 | 1 | 1 | 1 | n/a | 8 | 33522 | 747 |
105+
| DGX-B300 | 8 | MXFP8 | 512 | 4 | 8192 | 1 | 1 | 1 | n/a | 8 | 37316 | 831 |
106+
| DGX-H100 | 16 | FP8 | 1024 | 1 | 8192 | 1 | 1 | 1 | n/a | 8 | 14719 | 328 |
107+
108+
#### Model: Nemotron_3_Super
109+
110+
| System | #-GPUs | Precision | GBS | MBS | Sequence Length | TP | PP | CP | VP | EP | Tokens / sec / GPU | Model TFLOP / sec / GPU |
111+
|--------|--------|-----------|-----|-----|-----------------|----|----|----|----|----|-----------------------|-------------------------|
112+
| DGX-GB300 | 64 | MXFP8 | 512 | 1 | 8192 | 1 | 1 | 1 | n/a | 64 | 9652 | 817 |
113+
| DGX-GB300 | 64 | NVFP4 | 512 | 1 | 8192 | 1 | 1 | 1 | n/a | 64 | 9900 | 839 |
114+
| DGX-GB200 | 64 | MXFP8 | 512 | 1 | 8192 | 2 | 1 | 1 | n/a | 64 | 6742 | 571 |
115+
| DGX-GB200 | 64 | NVFP4 | 512 | 1 | 8192 | 2 | 1 | 1 | n/a | 64 | 6928 | 587 |
116+
| DGX-B300 | 64 | MXFP8 | 512 | 1 | 8192 | 1 | 1 | 1 | n/a | 8 | 7867 | 667 |
117+
| DGX-B300 | 64 | NVFP4 | 512 | 1 | 8192 | 1 | 1 | 1 | n/a | 8 | 8131 | 689 |
118+
119+
### SFT Performance
120+
121+
#### Model: LLAMA3_70B
122+
123+
| System | #-GPUs | Precision | GBS | MBS | Sequence Length | TP | PP | CP | VP | EP | Tokens / sec / GPU | Model TFLOP / sec / GPU |
124+
|--------|--------|-----------|-----|-----|-----------------|----|----|----|----|----|-----------------------|-------------------------|
125+
| DGX-GB300 | 32 | FP8 | 32 | 1 | 4096 | 1 | 2 | 1 | 20 | n/a | 4819 | 2083 |
126+
| DGX-GB300 | 32 | MXFP8 | 32 | 1 | 4096 | 1 | 2 | 1 | 20 | n/a | 4312 | 1877 |
127+
| DGX-GB200 | 32 | FP8 | 32 | 1 | 4096 | 1 | 8 | 1 | 10 | n/a | 3864 | 1671 |
128+
| DGX-GB200 | 32 | MXFP8 | 32 | 1 | 4096 | 1 | 8 | 1 | 10 | n/a | 3593 | 1553 |
129+
| DGX-H100 | 32 | FP8 | 32 | 1 | 4096 | 4 | 4 | 1 | 5 | n/a | 1638 | 710 |
130+
131+
### LoRA Performance
132+
133+
#### Model: LLAMA3_70B
134+
135+
| System | #-GPUs | Precision | GBS | MBS | Sequence Length | TP | PP | CP | VP | EP | Tokens / sec / GPU | Model TFLOP / sec / GPU |
136+
|--------|--------|-----------|-----|-----|-----------------|----|----|----|----|----|-----------------------|-------------------------|
137+
| DGX-GB300 | 8 | FP8 | 32 | 1 | 4096 | 1 | 2 | 1 | 20 | n/a | 7481 | 2086 |
138+
| DGX-GB300 | 8 | MXFP8 | 32 | 1 | 4096 | 1 | 2 | 1 | 20 | n/a | 7447 | 2072 |
139+
| DGX-GB200 | 8 | FP8 | 32 | 1 | 4096 | 1 | 2 | 1 | 20 | n/a | 6206 | 1731 |
140+
| DGX-GB200 | 8 | MXFP8 | 32 | 1 | 4096 | 1 | 4 | 1 | 20 | n/a | 5958 | 1663 |
141+
| DGX-H100 | 8 | FP8 | 32 | 1 | 4096 | 2 | 4 | 1 | 20 | n/a | 2643 | 735 |
142+
42143
## 26.04.01 NeMo Container
43144

44145
### Pre-Training Performance

0 commit comments

Comments
 (0)