Discuss PEFT broad-target semantics: match_all_linear vs target_modules ["*"]

[yuhezhang-ai]

Context

AutoModel currently has two related ways to select broad LoRA targets:

• match_all_linear=True is type-based. It matches ordinary torch.nn.Linear and Transformer Engine Linear modules, including lm_head, but does not match custom grouped/fused expert containers.
• target_modules: ["*"] wildcard-matches every module name. apply_lora_to_linear_modules then limits patching to supported LoRA target types, so it includes ordinary/TE linear modules and supported GroupedExperts implementations.

This distinction is not obvious from the name match_all_linear. Grouped/fused experts perform linear projections internally, so excluding them may be viewed either as useful standard-linear-only behavior or as legacy behavior from before grouped-expert/EP support.

There are currently eight active example YAMLs using match_all_linear: true, so changing or removing it would require an explicit compatibility and recipe-migration decision.

Verified behavior

This discussion is not prompted by a MiniMax reload failure. MiniMax M2.7 with target_modules: ["*"] already works end to end in scoped pipeline 61590436:

• train_and_save: PASS
• automodel_reload: PASS
• hf_reload: PASS
• Vanilla Hugging Face PEFT exact fingerprint comparison: 746 saved adapter tensors matched

Pipeline: https://gitlab-master.nvidia.com/dl/JoC/nemo-ci/-/pipelines/61590436

The metadata guard in #3439 addresses a separate invariant: expert target_parameters should only be emitted when expert LoRA was actually injected.

Design options

Option A: Retain match_all_linear with clearer documentation

Keep the existing standard-linear-only behavior and document it explicitly. Explain that grouped/fused expert containers are excluded, and that target_modules: ["*"] selects all supported LoRA target types, including supported grouped experts.

Potential benefits:

• Preserves existing Python/YAML behavior.
• Retains a useful way to target standard Linear types without automatically selecting grouped experts.
• Avoids unexpectedly selecting unsupported expert implementations.

Potential costs:

• Two broad-selection mechanisms remain.
• The name can continue to be interpreted as including linear-like expert projections.

Option B: Deprecate and remove match_all_linear

Make the wildcard form the canonical broad-selection configuration and migrate shipped recipes to:

target_modules: ["*"]

Potential benefits:

• One canonical broad-selection mechanism.
• Grouped experts are included consistently where LoRA support exists.
• Removes the ambiguity around what counts as a "linear" module.

Potential costs:

• Existing user configs and eight active example recipes need migration.
• On MoE models, migration can silently expand the trained parameter set to grouped experts, changing memory use and training behavior.
• Wildcard selection can reach unsupported expert implementations and raise NotImplementedError.
• Removal needs a deprecation window rather than an immediate compatibility break.

Option C: Replace it with a clearer name

Introduce a name such as match_all_standard_linear, retain match_all_linear temporarily as a deprecated alias, and separately document target_modules: ["*"] as all supported target types.

Potential benefits:

• Makes the current type-based contract explicit.
• Preserves the standard-linear-only use case.

Potential costs:

• Temporarily expands the public API and still requires migration/deprecation work.
• Does not reduce the number of concepts users must understand.

Option D: Broaden match_all_linear semantics

Keep the existing name but make it include supported grouped/fused expert projections.

Potential benefits:

• Aligns the name with a mathematical interpretation of all linear projections.

Potential costs:

• Silently changes the trained parameter set for existing MoE recipes and user configs.
• Becomes close to target_modules: ["*"], leaving two nearly equivalent mechanisms.

Questions for discussion

• Is standard-Linear-only targeting a useful public contract that should remain supported?
• Should the canonical "all supported LoRA targets" configuration be target_modules: ["*"]?
• If match_all_linear is removed or renamed, what deprecation period and migration warning should be provided?
• Should shipped MoE recipes migrate automatically to grouped-expert LoRA, or should that remain an explicit recipe choice because it changes training behavior?
• How should broad selection behave when a model contains an unsupported expert implementation such as GroupedExpertsTE?

Outcome

Use this issue to agree on the intended public contract first. After that decision, a follow-up PR can implement the chosen documentation, compatibility path, YAML migration, and focused regression tests.

Related work

• fix(peft): enable v5 expert adapters for Nemotron and MiniMax #3439
• Expert LoRA in Transformers v5 #1151

[yuhezhang-ai]

@akoumpa see the issue, in shorter words, I found our match_all_linear doesn't match the grouped experts layers (which is also linear layer, but not plain nn.Linear), this might be a bug.
If match_all_linear intentionally only match torch.nn.Linear but not grouped experts, then we should documented it clearly or maybe rename it.
If it's a bug, then we can consider fix this function, or remove it and just use target_modules: ["*"] to match all linear layers.

[akoumpa]

Hi @yuhezhang-ai , i think this is a bug, but also, we may want to deprecate match_all_linear. in practice, it will match the lm_head, then vLLM won't be able to load it because it doesn't support LoRA there. So i think this option is bad UX overall.