Skip to content

Commit 3b5c054

Browse files
authored
Updated Ultra Benchmark results (#116)
Signed-off-by: asurla <asurla@nvidia.com>
1 parent 736f8be commit 3b5c054

1 file changed

Lines changed: 8 additions & 8 deletions

File tree

  • usage-cookbook/Nemotron-3-Ultra-Base

usage-cookbook/Nemotron-3-Ultra-Base/README.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -20,16 +20,16 @@ It is designed to be the best possible **starting point for customization**: fin
2020

2121
Measured on **NVIDIA GB200 NVL72** against GLM-4.5-355B-A32B and Kimi-K2-1026B-A33B, Nemotron 3 Ultra base model delivers up to 5x higher TPS at max throughput and leading accuracy for various agentic tasks:
2222

23-
| Benchmark | Ultra 550B-A55B | GLM-4.5-355B-A32B (reported) | Kimi-K2-1026B-A33B |
23+
| Benchmark | Ultra 550B-A55B | GLM-4.5-355B-A32B | Kimi-K2-1026B-A33B |
2424
| ----- | ----- | ----- | ----- |
25-
| MMLU Pro | **78.0** | 65.6 | 69.3 |
26-
| MMLU | **89.3** | 86.3 | 88.0 |
27-
| Code | **86.1** | 76.2 | 75.3 |
28-
| Math | **86.0** | 72.1 | 79.5 |
29-
| Common Sense | 81.1 | 81.3 | **81.6** |
30-
| Multilingual | **89.1** | 83.3 | 84.2 |
25+
| MMLU Pro | **79.0** | 65.6 | 69.3 |
26+
| MMLU | **89.1** | 86.3 | 88.0 |
27+
| Code | **85.3** | 76.2 | 75.3 |
28+
| Math | **85.4** | 72.1 | 79.5 |
29+
| Common Sense | 81.0 | 81.3 | **81.6** |
30+
| Multilingual | **89.0** | 83.3 | 84.2 |
3131
| Peak Throughput | **** vs GLM || \~2.5× |
3232

3333
## **Availability**:
3434

35-
Weights will become available with the full release of Nemotron 3 Ultra, expected to release in 1H 2026\.
35+
Weights will become available with the full release of Nemotron 3 Ultra, expected to release in 1H 2026\.

0 commit comments

Comments
 (0)