[Bug] Nemotron 3 Ultra-550B-A55B-NVFP4 on 8xH100: error in cookbook + unsupported feature on sm_90a

[karna48]

Describe the Bug

The cookbook is wrong for 8xH100,
tensor paralelism must be 8, and no data paralelism. It seems untested.
https://github.com/NVIDIA-NeMo/Nemotron/blob/main/usage-cookbook/Nemotron-3-Ultra/vllm_cookbook.ipynb

However after the correction error is encountered:

Reproduction Steps

1. Run vllm serve with parameters:

nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 \
 --served-model-name nemotron-3-ultra \
 --trust-remote-code \
 --tensor-parallel-size 8 \
 --mamba-cache-dtype float16 \
 --mamba-ssm-cache-dtype float16 \
 --mamba-cache-philox-rounds 5 \
 --enable-mamba-cache-stochastic-rounding \
 --max-num-seqs 128 \
 --tool-call-parser qwen3_coder \
 --reasoning-parser nemotron_v3 \
 --host localhost \
 --port 45005 

Expected Behavior

vLLM to serve the model

Actual Behavior

PTX generation/compilation error

Environment

• OS: Ubuntu 22.04.5 LTS (Base Command Manager 10.0, SLURM, Enroot)
• Python version: 3.12.13 (in vllm/vllm-openai:v0.22.0 container)
• Nemotron version: Nemotron 3 Ultra-550B-A55B-NVFP4
• GPU/CUDA: H100 / CUDA 13.0
• Relevant dependencies:

Additional Context

slurm-16095.txt

[karna48]

The problematic feature was -mamba-cache-stochastic-rounding .
These parameters work on 8xH100:

--served-model-name nemotron-3-ultra \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --data-parallel-size 1 \
  --mamba-backend triton \
  --mamba-ssm-cache-dtype float32 \
  --mamba-cache-philox-rounds 5 \
  --max-num-seqs 128 \
  --tool-call-parser qwen3_coder \
  --reasoning-parser nemotron_v3