Merge main and address Kimi-K3 review feedback

Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>

Author: Edwardf0t1

.github/CODEOWNERS

@@ -63,5 +63,8 @@ modelopt_recipes @NVIDIA/modelopt-recipes-codeowners
 /examples/vllm_serve @NVIDIA/modelopt-examples-llm_ptq-codeowners
 /examples/windows @NVIDIA/modelopt-windows-codeowners
 
+# Agent plugin (skills, agent config)
+/plugins @NVIDIA/modelopt-agents-codeowners
+
 # Requirements files are owned by the setup team regardless of location
 requirements*.txt @NVIDIA/modelopt-setup-codeowners

CHANGELOG.rst

@@ -9,6 +9,7 @@ Changelog
 *Quantization*
 
 - Add a calibration-free streaming Kimi-K3 converter and checkpoint-mirror recipe for NVFP4 routed experts with ``input_scale=1.0`` and 128x128 block-FP8 KDA/MLA attention weights. The converter operates shard-by-shard on the source checkpoint's packed MXFP4 experts instead of loading the 2.8T model through the in-memory ``hf_ptq.py`` path.
+- Add ``mtq.temporarily_fold_weights`` for repeated frozen-weight inference and ``mtq.preserve_quantizer_attributes_context`` for restoring temporary quantizer property and type changes. Temporary folding snapshots affected fake-quant weights on a configurable device and restores them with their quantizer state; retained pre-quant scales are inactive, while shared weights, shared quantizers, and ``SequentialQuantizer`` weights are unsupported.
 - Add the ``nvfp4_act_headroom`` calibration algorithm for NVFP4 **activation** global scales. Instead of setting the global scale from the largest per-block amax seen during calibration (plain ``max``, which leaves no room above it so any larger activation saturates), it anchors the scale to a low percentile of the per-block amax distribution, leaving the rest of the FP8 block-scale range as headroom: ``amax = max(rho * anchor, upper)``, where ``anchor`` and ``upper`` are the per-block amaxes at ``anchor_percentile`` (default 1) and ``upper_percentile`` (default 99.99; set to 100 to never clip calibration data), and ``rho`` (default 16384) is the headroom factor. Applies only to NVFP4 dynamic-block input quantizers; ``SequentialQuantizer`` activation quantizers raise. Weight scales are an orthogonal axis selected by a nested ``weight_scale_algorithm`` (``max`` by default, or ``mse`` / ``local_hessian``), so one recipe can combine a weight calibration with this activation policy in a single pass. Ships ``modelopt_recipes/general/ptq/nvfp4_act_headroom-kv_fp8_cast.yaml``, which mirrors ``nvfp4_default-kv_fp8_cast`` with only the calibration algorithm swapped and exports a standard NVFP4 checkpoint.
 
 *Megatron Framework (M-LM / M-Bridge)*
@@ -26,6 +27,12 @@ Changelog
 **Backward Breaking Changes**
 
 - Move the Mistral Medium 3.5 checkpoint-mirror recipe from ``huggingface/models/nvidia/Mistral-Medium-3.5-128B-NVFP4/ptq/nvfp4-max-calib`` to ``huggingface/models/mistralai/Mistral-Medium-3.5-128B/ptq/nvfp4-max-calib``, keying it by the canonical Hugging Face base model. Update any saved ``--recipe`` paths to the new location.
+- Remove the ``--auto_quantize_bits``, ``--auto_quantize_method``, ``--auto_quantize_score_size``, ``--auto_quantize_cost_model`` and ``--auto_quantize_active_moe_expert_ratio`` flags from ``examples/hf_ptq`` (deprecated in 0.46). Use an AutoQuantize ``--recipe`` from ``modelopt_recipes/general/auto_quantize/`` instead. Those recipes now also splice in the shared base ``cost_excluded_layers`` unit, which the removed CLI applied unconditionally, so a VL model keeps its vision tower and MTP layers out of the effective-bits denominator. On a VL model this changes the per-layer cost weights, so an existing ``--auto_quantize_checkpoint`` from an earlier release is rejected with "Use a different checkpoint path"; delete or repoint it to re-run the search.
+- Remove the ``examples/llm_ptq`` symlink and the ``examples/vlm_ptq`` forwarder (both deprecated in 0.46). Use ``examples/hf_ptq``, passing ``--vlm`` for vision-language models.
+- Remove the backward-compat ``--qformat`` / ``--quant_cfg`` short names ``int8_sq``, ``int8_wo``, ``w4a8_awq``, ``nvfp4_awq``, ``nvfp4_mse``, ``nvfp4_local_hessian``, ``fp8_pb_wo`` and ``fp8_pc_pt`` (deprecated in 0.45). Use the preset basename under ``modelopt_recipes/configs/ptq/presets/model/`` instead: ``int8_smoothquant``, ``int8_weight_only``, ``w4a8_awq_beta``, ``nvfp4_awq_lite``, ``nvfp4_w4a4_weight_mse_fp8_sweep``, ``nvfp4_w4a4_weight_local_hessian``, ``fp8_2d_blockwise_weight_only`` and ``fp8_per_channel_per_token``. The ``modelopt.recipe.presets.QFORMAT_ALIASES`` table and the ``aliases`` argument of ``load_quant_cfg_choices()`` are removed along with them.
+- Remove the legacy ``layerwise`` bool form, its ``use_sequential`` alias, and the top-level ``layerwise_checkpoint_dir`` key from calibration algorithm configs (deprecated in 0.45). Use the nested form, e.g. ``layerwise: {enable: true, checkpoint_dir: /path}``. A pre-0.45 ``modelopt_state`` carrying either legacy key now fails validation on restore instead of being migrated; re-save it with a 0.45/0.46 release first.
+- Remove in-trainer quantization via ``QuantizationArguments.quant_cfg`` / ``--quant_cfg`` (deprecated in 0.45); use ``--recipe``. New recipes ``general/ptq/mxfp4_mlp_weight_only`` and ``general/ptq/nvfp4_mlp_weight_only`` replace ``MXFP4_MLP_WEIGHT_ONLY_CFG`` / ``NVFP4_MLP_WEIGHT_ONLY_CFG`` in the ``examples/gpt-oss`` QAT flow.
+- Remove the ``QuantizationArgumentsWithConfig`` alias in ``modelopt.torch.quantization.plugins.transformers_trainer`` (deprecated in 0.45). Use ``QuantizationArguments``.
 - Transformer Engine ``TEGroupedMLP`` (fused MoE experts) now uses **per-expert** weight quantization (one ``amax`` per expert) instead of a single shared ``amax``, so ModelOpt checkpoints containing quantized ``TEGroupedMLP`` modules saved before 0.47 are **not compatible** with 0.47. Re-run PTQ to regenerate compatible checkpoints.
 
 **Deprecations**
@@ -37,6 +44,7 @@ Changelog
 - Avoid querying CUDA/Blackwell capability when ``NVFP4QTensor.quantize`` uses its CPU path or has the optional TensorRT-LLM fast path disabled.
 - Update HuggingFace checkpoint export to use name-based tied-weight deduplication instead of the previous address-based approach. The address-based deduplication could incorrectly drop an untied weight that happened to share memory with a tied one, producing an incomplete checkpoint (observed as a false positive on MiniMax-M2.7).
 - Fix EAGLE-3 training with context parallelism (``--cp_size > 1`` in ``examples/speculative_decoding``), which failed to start on ``accelerate >= 1.13`` and then raised ``got mixed torch.Tensor and DTensor``.
+- Polygraphy minimum dependency upgraded to ``0.53.4`` to solve ONNX AutoCast failures when marking optional graph outputs.
 
 0.46 (2026-08-17)
 ^^^^^^^^^^^^^^^^^

examples/gpt-oss/README.md

@@ -57,11 +57,12 @@ If you are training Huggingface models with trainer classes from Huggingface suc
 A real end-to-end example for this is in `sft.py` in this folder. To perform QAT with full parameter SFT on GPT-OSS 20B model, run:
 
 ```sh
-# Other supported quantization configs include NVFP4_MLP_WEIGHT_ONLY_CFG, NVFP4_MLP_ONLY_CFG etc.
+# Other supported quantization recipes include general/ptq/nvfp4_mlp_weight_only, or
+# general/ptq/nvfp4_mlp_only-kv_fp8 (also quantizes activations and the KV cache to FP8, which needs calibration).
 # [Optional] For faster FlashAttention3, add '--attn_implementation kernels-community/vllm-flash-attn3'
 accelerate launch --config_file configs/zero3.yaml sft.py \
     --config configs/sft_full.yaml --model_name_or_path openai/gpt-oss-20b \
-    --quant_cfg MXFP4_MLP_WEIGHT_ONLY_CFG \
+    --recipe general/ptq/mxfp4_mlp_weight_only \
     --output_dir gpt-oss-20b-qat
 ```
 
@@ -89,7 +90,7 @@ accelerate launch --config_file configs/zero3.yaml sft.py \
 # Step 2: Perform QAT on the high precision SFT checkpoint
 accelerate launch --config_file configs/zero3.yaml sft.py \
     --config configs/sft_full.yaml --model_name_or_path gpt-oss-20b-sft \
-    --quant_cfg MXFP4_MLP_WEIGHT_ONLY_CFG \
+    --recipe general/ptq/mxfp4_mlp_weight_only \
     --output_dir gpt-oss-20b-qat \
 ```
 
@@ -160,7 +161,7 @@ Here is how to run LoRA QAT for GPT OSS 120B model:
 ```bash
 python sft.py --config configs/sft_lora.yaml \
     --model_name_or_path openai/gpt-oss-120b \
-    --quant_cfg MXFP4_MLP_WEIGHT_ONLY_CFG \
+    --recipe general/ptq/mxfp4_mlp_weight_only \
     --output_dir gpt-oss-120b-lora-qat
 ```
 

examples/gpt-oss/configs/sft_full.yaml

@@ -30,6 +30,6 @@ eval_steps: 8
 dataset_test_split: test
 
 # ModelOpt Quantization Parameters
-quant_cfg: # Examples: MXFP4_MLP_WEIGHT_ONLY_CFG, NVFP4_MLP_WEIGHT_ONLY_CFG, NVFP4_MLP_ONLY_CFG
-  # For the full list of supported configs, do: mtq.config.choices
+recipe: # Examples: general/ptq/mxfp4_mlp_weight_only, general/ptq/nvfp4_mlp_weight_only
+  # For the full list of built-in recipes, see modelopt_recipes/general/ptq/
 calib_size: 128

examples/gpt-oss/configs/sft_lora.yaml

@@ -35,6 +35,6 @@ eval_steps: 8
 dataset_test_split: test
 
 # ModelOpt Quantization Parameters
-quant_cfg: # Examples: MXFP4_MLP_WEIGHT_ONLY_CFG, NVFP4_MLP_WEIGHT_ONLY_CFG, NVFP4_MLP_ONLY_CFG
-  # For the full list of supported configs, do: mtq.config.choices
+recipe: # Examples: general/ptq/mxfp4_mlp_weight_only, general/ptq/nvfp4_mlp_weight_only
+  # For the full list of built-in recipes, see modelopt_recipes/general/ptq/
 calib_size: 128

examples/gpt-oss/sft.py

@@ -37,7 +37,7 @@
     --packing true packing_strategy wrapped \
     --run_name 20b-full-qat \
     --attn_implementation kernels-community/vllm-flash-attn3
-    --quant_cfg MXFP4_MLP_WEIGHT_ONLY_CFG
+    --recipe general/ptq/mxfp4_mlp_weight_only
 """
 
 from transformers import AutoModelForCausalLM, AutoTokenizer, Mxfp4Config

examples/hf_ptq/README.md

@@ -97,7 +97,7 @@ Please reference our [framework scripts](#framework-scripts) and our [docs](http
 
 ### Hugging Face Supported Models
 
-| Model | fp8 | int8_sq | int4_awq | w4a8_awq<sup>1</sup> | nvfp4<sup>5</sup> |
+| Model | fp8 | int8_smoothquant | int4_awq | w4a8_awq_beta<sup>1</sup> | nvfp4<sup>5</sup> |
 | :---: | :---: | :---: | :---: | :---: | :---: |
 | LLAMA 3.x | ✅ | ❌ | ✅ | ✅<sup>3</sup> | ✅ |
 | LLAMA 4 <sup>6</sup> | ✅ | ❌ | ❌ | ❌ | ✅ |
@@ -126,7 +126,7 @@ Please reference our [framework scripts](#framework-scripts) and our [docs](http
 
 > *This is a subset of the models supported. For the full list please check the [TensorRT-LLM support matrix](https://nvidia.github.io/TensorRT-LLM/reference/precision.html#support-matrix)*
 
-> *<sup>1.</sup>The w4a8_awq is an experimental quantization scheme that may result in a higher accuracy penalty.* \
+> *<sup>1.</sup>The w4a8_awq_beta is an experimental quantization scheme that may result in a higher accuracy penalty.* \
 > *<sup>2.</sup>For some models, there is only support for exporting quantized checkpoints.* \
 > *<sup>3.</sup>W4A8_AWQ is only available on some models but not all* \
 > *<sup>4.</sup>For some models, KV cache quantization may result in a higher accuracy penalty.* \
@@ -137,7 +137,7 @@ Please reference our [framework scripts](#framework-scripts) and our [docs](http
 > *<sup>9.</sup>Running Whisper model with transformers>=5.0 requires [torchcodec](https://github.com/meta-pytorch/torchcodec?tab=readme-ov-file#installing-cuda-enabled-torchcodec) and other system packages (e.g. ffmpeg).* \
 > *<sup>10.</sup>GPT-OSS ships with native MXFP4 weights; NVFP4 export is produced via the closed-form `--cast_mxfp4_to_nvfp4` cast (see [MXFP4 → NVFP4 cast](#mxfp4--nvfp4-cast-for-gpt-oss)).* \
 > *<sup>11.</sup>Vision-language model (VLM): only the language model is quantized while the vision encoder is kept in high precision. Pass `--vlm` to the shell script (see [VLM quantization](#vlm-quantization)).* \
-> *<sup>12.</sup>For VLMs, `int8_sq` only supports TensorRT-LLM checkpoint export and is not compatible with the TensorRT-LLM torch backend.* \
+> *<sup>12.</sup>For VLMs, `int8_smoothquant` only supports TensorRT-LLM checkpoint export and is not compatible with the TensorRT-LLM torch backend.* \
 > *<sup>13.</sup>Nemotron VL automatically calibrates with image-text pairs; see [VLM calibration with image-text pairs](#vlm-calibration-with-image-text-pairs-eg-nemotron-vl).* \
 > *<sup>14.</sup>Kimi K3 uses the calibration-free [streaming converter](../kimi/README.md) because its routed experts are released as packed MXFP4 tensors; it does not use the in-memory `hf_ptq.py` flow.*
 
@@ -164,7 +164,7 @@ export HF_PATH=<the downloaded LLaMA checkpoint from the Hugging Face hub, or si
 scripts/huggingface_example.sh --model $HF_PATH --quant <QFORMAT> --tp [1|2|4|8]
 ```
 
-Supported `QFORMAT` values: `fp8`, `fp8_pc_pt`, `fp8_pb_wo`, `int8`, `int8_sq`, `int8_wo`, `int4_awq`, `w4a8_awq`, `nvfp4`, `nvfp4_awq`, `nvfp4_mse`, `nvfp4_mlp_only`, `nvfp4_experts_only`, `nvfp4_omlp_only`, `nvfp4_svdquant`, `nvfp4_local_hessian`, `w4a8_nvfp4_fp8`, `w4a8_mxfp4_fp8`, `mxfp8`.
+`QFORMAT` accepts any preset basename under [`modelopt_recipes/configs/ptq/presets/model/`](../../modelopt_recipes/configs/ptq/presets/model) — e.g. `fp8`, `fp8_per_channel_per_token`, `fp8_2d_blockwise_weight_only`, `int8`, `int8_smoothquant`, `int8_weight_only`, `int4_awq`, `w4a8_awq_beta`, `nvfp4`, `nvfp4_awq_lite`, `nvfp4_w4a4_weight_mse_fp8_sweep`, `nvfp4_mlp_only`, `nvfp4_experts_only`, `nvfp4_omlp_only`, `nvfp4_svdquant`, `nvfp4_w4a4_weight_local_hessian`, `w4a8_nvfp4_fp8`, `w4a8_mxfp4_fp8`, `mxfp8`.
 
 > *By default `trust_remote_code` is set to false. Please turn it on if model calibration and eval requires it using `--trust_remote_code`.*
 
@@ -273,10 +273,8 @@ TensorRT-LLM multimodal quickstart as the deploy smoke test instead of the text-
 scripts/huggingface_example.sh --model <Hugging Face model card or checkpoint> --quant fp8 --vlm
 ```
 
-Supported `--quant` values for VLMs are `fp8`, `nvfp4`, `int8_sq`, `int4_awq`, and `w4a8_awq` (see
-the `(VLM)` rows in the [Support Matrix](#hugging-face-supported-models)).
-
-> *This consolidates the former `examples/vlm_ptq` example, which now forwards here.*
+Supported `--quant` values for VLMs are `fp8`, `nvfp4`, `int8_smoothquant`, `int4_awq`, and
+`w4a8_awq_beta` (see the `(VLM)` rows in the [Support Matrix](#hugging-face-supported-models)).
 
 #### VLM calibration with image-text pairs (e.g., Nemotron VL)
 
@@ -367,17 +365,6 @@ search-disabled layers, and cost-excluded layers — see
 recipes (carrying architecture-specific disabled layers — e.g. VL vision towers) live under
 `modelopt_recipes/huggingface/<model>/auto_quantize/`.
 
-> *Migration: prefer an AutoQuantize `--recipe`. The `--auto_quantize_bits`, `--auto_quantize_method`,
-> `--auto_quantize_score_size`, `--auto_quantize_cost_model`, and `--auto_quantize_active_moe_expert_ratio`
-> CLI flags are **deprecated but still work** — they are converted into an `AutoQuantizeConfig` on the fly
-> (with a `DeprecationWarning`) and will be removed in a future release. They map to recipe fields:
-> `--auto_quantize_bits` → `constraints.effective_bits`, `--auto_quantize_method` → `auto_quantize_method`,
-> `--auto_quantize_score_size` → `score_size`, `--auto_quantize_cost_model` → `constraints.cost_model`,
-> `--auto_quantize_active_moe_expert_ratio` → `constraints.cost.active_moe_expert_ratio`, and the
-> `--qformat fp8,nvfp4` candidate list → `candidate_formats`. When converted, the shared base
-> `disabled_layers` and `cost_excluded_layers` patterns are appended automatically. `--auto_quantize_checkpoint`
-> is unchanged. Start from a shipped recipe under `modelopt_recipes/general/auto_quantize/`.*
-
 [Script](./scripts/huggingface_example.sh)
 
 ```bash

examples/hf_ptq/example_utils.py

@@ -1134,37 +1134,30 @@ def copy_custom_model_files(
         print("No checkpoint sidecar files found to copy")
 
 
-def _layerwise_checkpoint_dir_location(algorithm) -> tuple[str, str] | None:
-    """Return ``("flat"/"nested", checkpoint_dir)`` for the layerwise checkpoint dir, or None."""
+def _layerwise_checkpoint_dir(algorithm) -> str | None:
+    """Return the nested ``layerwise.checkpoint_dir``, or None."""
     if not isinstance(algorithm, dict):
         return None
-    flat = algorithm.get("layerwise_checkpoint_dir")
-    if flat is not None:
-        return "flat", flat
     nested = algorithm.get("layerwise") or {}
-    ckpt = nested.get("checkpoint_dir") if isinstance(nested, dict) else None
-    return ("nested", ckpt) if ckpt is not None else None
+    return nested.get("checkpoint_dir") if isinstance(nested, dict) else None
 
 
 def needs_checkpoint_path_update(quant_cfg: dict) -> bool:
     """Check if quant_cfg has a layerwise checkpoint_dir that should be auto-resolved to a unique subpath."""
-    return _layerwise_checkpoint_dir_location(quant_cfg.get("algorithm")) is not None
+    return _layerwise_checkpoint_dir(quant_cfg.get("algorithm")) is not None
 
 
 def resolve_checkpoint_dir(quant_cfg: dict, model_path: str) -> tuple[dict, str]:
     """Append a unique ``<model_name>_<config_hash>`` subdirectory to the layerwise checkpoint_dir.
 
     Allows a single recipe to be reused across models without checkpoint collisions.
-    Supports both the legacy flat ``layerwise_checkpoint_dir`` and the nested
-    ``layerwise.checkpoint_dir`` shape, writing back to whichever the user provided.
     Must only be called when :func:`needs_checkpoint_path_update` returns True.
 
     Returns ``(updated_quant_cfg, resolved_path)`` so the caller can log or
     reference the resolved path without re-deriving the dict shape.
     """
-    location = _layerwise_checkpoint_dir_location(quant_cfg["algorithm"])
-    assert location is not None  # guaranteed by needs_checkpoint_path_update
-    shape, base_dir = location
+    base_dir = _layerwise_checkpoint_dir(quant_cfg["algorithm"])
+    assert base_dir is not None  # guaranteed by needs_checkpoint_path_update
 
     name = model_path.rstrip("/")
     if "/" in name and not os.path.isabs(name):
@@ -1176,11 +1169,7 @@ def resolve_checkpoint_dir(quant_cfg: dict, model_path: str) -> tuple[dict, str]
     resolved = os.path.join(base_dir, f"{name}_{config_hash}")
 
     quant_cfg = copy.deepcopy(quant_cfg)
-    algo = quant_cfg["algorithm"]
-    if "layerwise_checkpoint_dir" in algo:
-        algo["layerwise_checkpoint_dir"] = resolved
-    if isinstance(algo.get("layerwise"), dict) and "checkpoint_dir" in algo["layerwise"]:
-        algo["layerwise"]["checkpoint_dir"] = resolved
+    quant_cfg["algorithm"]["layerwise"]["checkpoint_dir"] = resolved
     return quant_cfg, resolved