Commit bbd7d02
Import the MTP block's final_layernorm (repeated-MTP path)
The repeated-MTP import branch loops over the inner MTP layers but never
imports the MTP block's own final_layernorm, leaving it random-initialized.
Add the import after the loop, keyed to the last inner layer index
(layer_id - 1) so it round-trips with the export side, which writes
mtp.layers.{last}.final_layernorm.
Bug found by @jenchen13.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>1 parent 89ef596 commit bbd7d02
1 file changed
Lines changed: 5 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
860 | 860 | | |
861 | 861 | | |
862 | 862 | | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
| 866 | + | |
| 867 | + | |
863 | 868 | | |
864 | 869 | | |
865 | 870 | | |
| |||
0 commit comments