Skip to content

Commit f3e88df

Browse files
authored
Move Mistral Medium recipe under canonical base model (#2150)
### What does this PR do? Type of change: Bug fix Moves the Mistral Medium 3.5 checkpoint-mirror PTQ recipe under the canonical Hugging Face base model, `mistralai/Mistral-Medium-3.5-128B`. Updates the recipe catalog, loader smoke-test path, and changelog. ### Usage ```bash python examples/hf_ptq/hf_ptq.py \ --model mistralai/Mistral-Medium-3.5-128B \ --recipe huggingface/models/mistralai/Mistral-Medium-3.5-128B/ptq/nvfp4-max-calib ``` ### Testing - `uvx --from pre-commit pre-commit run --files CHANGELOG.rst modelopt_recipes/huggingface/models/mistralai/Mistral-Medium-3.5-128B/ptq/nvfp4-max-calib.yaml modelopt_recipes/ptq.md tests/unit/recipe/test_loader.py` - Recipe documentation consistency checks (all passed) - Focused pytest was unavailable locally because the checkout environment lacks pytest/Torch; the dedicated pre-commit recipe validator passed. ### Before your PR is "*Ready for review*" - Is this change backward compatible?: ❌ The built-in recipe path changes; the changelog documents the replacement path. - If you copied code from any other sources or added a new PIP dependency, did you follow guidance in `CONTRIBUTING.md`: N/A - Did you write any new necessary tests?: ✅ Updated the built-in recipe loader smoke test. - Did you update [Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?: ✅ - Did you get Claude approval on this PR?: N/A ### Additional Information The recipe targets the vendor's canonical base checkpoint, [`mistralai/Mistral-Medium-3.5-128B`](https://huggingface.co/mistralai/Mistral-Medium-3.5-128B), and reproduces NVIDIA's [`Mistral-Medium-3.5-128B-NVFP4`](https://huggingface.co/nvidia/Mistral-Medium-3.5-128B-NVFP4) quantization map. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a Mistral Medium 3.5 128B post-training quantization recipe supporting NVFP4 and FP8 configurations with max calibration. * **Documentation** * Updated checkpoint-mirror guidance to reflect the new model-based recipe location. * **Breaking Changes** * Saved references to the previous recipe path must be updated to the new Mistral Medium 3.5 recipe path. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Signed-off-by: Chad Voegele <cvoegele@nvidia.com>
1 parent c15d2b5 commit f3e88df

4 files changed

Lines changed: 5 additions & 4 deletions

File tree

CHANGELOG.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ Changelog
2222

2323
**Backward Breaking Changes**
2424

25+
- Move the Mistral Medium 3.5 checkpoint-mirror recipe from ``huggingface/models/nvidia/Mistral-Medium-3.5-128B-NVFP4/ptq/nvfp4-max-calib`` to ``huggingface/models/mistralai/Mistral-Medium-3.5-128B/ptq/nvfp4-max-calib``, keying it by the canonical Hugging Face base model. Update any saved ``--recipe`` paths to the new location.
2526
- Transformer Engine ``TEGroupedMLP`` (fused MoE experts) now uses **per-expert** weight quantization (one ``amax`` per expert) instead of a single shared ``amax``, so ModelOpt checkpoints containing quantized ``TEGroupedMLP`` modules saved before 0.47 are **not compatible** with 0.47. Re-run PTQ to regenerate compatible checkpoints.
2627

2728
**Deprecations**

modelopt_recipes/huggingface/models/nvidia/Mistral-Medium-3.5-128B-NVFP4/ptq/nvfp4-max-calib.yaml renamed to modelopt_recipes/huggingface/models/mistralai/Mistral-Medium-3.5-128B/ptq/nvfp4-max-calib.yaml

File renamed without changes.

modelopt_recipes/ptq.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -235,7 +235,7 @@ that baseline. The deviations come in four kinds:
235235
| **Architecture-aware `quant_cfg`** | Per-sub-module format choices a single wildcard scheme can't express | `minimax_m3_vl`, `qwen3_5`, `qwen3_5_moe`, `vit`, `nemotron_llama` |
236236
| **Algorithm override** | Same numerics & scope, but the *calibration algorithm* is tweaked because the default breaks or regresses | `gemma`, `gemma4`, `mpt` |
237237
| **Extra exclusions** | Adds disabled-quantizer patterns so non-language branches stay full precision | `nemotron_vl`, `diffusion_gemma` |
238-
| **Checkpoint mirror** | A mixed-precision map reproducing one published checkpoint exactly | `models/nvidia/Nemotron-3-*`, `models/nvidia/Mistral-Medium-3.5-128B-NVFP4` |
238+
| **Checkpoint mirror** | A mixed-precision map reproducing one published checkpoint exactly | `models/nvidia/Nemotron-3-*`, `models/mistralai/Mistral-Medium-3.5-128B` |
239239

240240
The numerics and standard exclusions are still inherited from `configs/`
241241
wherever possible — the model folder captures *only* the delta. Each `<task>/`
@@ -335,12 +335,12 @@ encoders (or the never-calibrated self-conditioning branch), regressing those
335335
modalities or crashing export. The extra patterns keep them in full precision;
336336
everything else matches the general recipe.
337337

338-
### Checkpoint mirrors — `models/nvidia/<checkpoint>`
338+
### Checkpoint mirrors — `models/<org>/<checkpoint>`
339339

340340
The `huggingface/models/` tier reproduces a **single published (or planned)
341341
checkpoint's** quant config verbatim:
342342

343-
- **`Mistral-Medium-3.5-128B-NVFP4/ptq/nvfp4-max-calib`** mirrors
343+
- **`models/mistralai/Mistral-Medium-3.5-128B/ptq/nvfp4-max-calib`** mirrors
344344
`nvidia/Mistral-Medium-3.5-128B-NVFP4`: decoder MLP layers 4–86 use NVFP4
345345
W4A4, edge MLP layers 0–3 and 87 use FP8 W8A8, and all attention projections
346346
and the KV cache use FP8. It uses max calibration.

tests/unit/recipe/test_loader.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -167,7 +167,7 @@ def test_load_recipe_builtin_description():
167167
"general/ptq/nvfp4_experts_only-kv_fp8",
168168
"general/ptq/nvfp4_experts_only-kv_fp8_cast",
169169
"general/ptq/nvfp4_experts_only-kv_fp8_layerwise",
170-
"huggingface/models/nvidia/Mistral-Medium-3.5-128B-NVFP4/ptq/nvfp4-max-calib",
170+
"huggingface/models/mistralai/Mistral-Medium-3.5-128B/ptq/nvfp4-max-calib",
171171
"general/ptq/nvfp4_mlp_only-kv_fp8",
172172
"general/ptq/nvfp4_mlp_only-novit-kv_fp8",
173173
"general/ptq/nvfp4_mlp_only-kv_fp8_cast",

0 commit comments

Comments
 (0)