lora training on qwen3_omni_moe works fine, but the saved adapter file has broken key names, so it can't be loaded with hf peft afterwards.
the adapter puts thinker. in front of every key on save, even peft keys that already start with base_model.model.. so you get
thinker.base_model.model.model.layers.0.self_attn.q_proj.lora_A.weight
but peft needs
base_model.model.thinker.model.layers.0.self_attn.q_proj.lora_A.weight
the weights themselves are all correct, only the prefix is in the wrong spot.
this happens in components/models/qwen3_omni_moe/state_dict_adapter.py, where to_hf and convert_single_tensor_to_hf add thinker. to every key without checking for the peft prefix. easy to see on cpu: call to_hf on a peft style state dict and every output key starts with thinker.base_model. instead of base_model..
resume from checkpoint still works, since from_hf strips the prefix back off. only the exported adapter file is broken.
same kind of bug as #3385 (kimi k25 vl, fixed in #3431), and the same fix should work here.
happy to send a pr for it.
lora training on qwen3_omni_moe works fine, but the saved adapter file has broken key names, so it can't be loaded with hf peft afterwards.
the adapter puts
thinker.in front of every key on save, even peft keys that already start withbase_model.model.. so you getbut peft needs
the weights themselves are all correct, only the prefix is in the wrong spot.
this happens in
components/models/qwen3_omni_moe/state_dict_adapter.py, whereto_hfandconvert_single_tensor_to_hfaddthinker.to every key without checking for the peft prefix. easy to see on cpu: callto_hfon a peft style state dict and every output key starts withthinker.base_model.instead ofbase_model..resume from checkpoint still works, since
from_hfstrips the prefix back off. only the exported adapter file is broken.same kind of bug as #3385 (kimi k25 vl, fixed in #3431), and the same fix should work here.
happy to send a pr for it.