Skip to content

Commit 506e04e

Browse files
committed
Only emit qk_norm_epsilon GQA attribute when q/k norm weights are present
The unconditional qk_norm_epsilon attribute broke plain GQA models (e.g. phi-2): the current onnxruntime nightly does not recognize the attribute, so model load failed with "Unrecognized attribute: qk_norm_epsilon for operator GroupQueryAttention". It also failed the test_plain_gqa_omits_qk_norm_epsilon_attribute unit test. Restore the guard so the attribute is only added on the fused Q/K norm path.
1 parent 7fc7837 commit 506e04e

1 file changed

Lines changed: 2 additions & 1 deletion

File tree

  • src/python/py/models/builders

src/python/py/models/builders/base.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2751,8 +2751,9 @@ def make_group_query_attention(self, name, **kwargs):
27512751
"softcap": self.attention_attrs["softcap"],
27522752
"do_rotary": self.attention_attrs["use_rope_in_attn"],
27532753
"rotary_interleaved": self.rope_attrs["interleaved"],
2754-
"qk_norm_epsilon": kwargs.get("qk_norm_epsilon", self.attention_attrs["qk_norm_epsilon"]),
27552754
}
2755+
if q_norm_weight:
2756+
attributes["qk_norm_epsilon"] = kwargs.get("qk_norm_epsilon", self.attention_attrs["qk_norm_epsilon"])
27562757
self.make_node(
27572758
"GroupQueryAttention",
27582759
inputs=inputs,

0 commit comments

Comments
 (0)