[Bug] GemmTransposeFusion folds an identity Transpose (perm=[0,1]) into Gemm transA/transB, producing silently wrong results

[lackjava-creator]

Describe the issue

Transpose with perm=[0,1] on a rank-2 tensor is the identity. When such a
node feeds a MatMul that is followed by Add, ORT fuses the pattern into a
single Gemm and sets transA=1 (or transB=1), so the fused node computes
A^T @ B + C instead of A @ B + C.

Consequences depend only on shapes:

• M == K (square operand): the model silently returns wrong numbers.
• M != K: execution fails with
  InvalidArgument ... Gemm: Invalid bias shape for broadcast,
  reported on a node named /MatMulAddFusion/GemmTransposeFusion/.

With ORT_DISABLE_ALL the same model always produces the correct result, so
this is purely a graph-optimization defect.

This is a regression: onnxruntime 1.20.1 is correct, 1.27.0 is not.

Root cause

onnxruntime/core/optimizer/gemm_transpose_fusion.cc decides to fold based on
the operator type alone and never inspects the perm attribute:

if (A_node_ptr != nullptr && A_node_ptr->OpType() == "Transpose") {
  ...
  transA = !transA;
}

Any Transpose toggles the flag, including a perm that is the identity
permutation. The fusion should only fire when perm == [1, 0] for rank-2
inputs (and should treat identity perm as a no-op that can simply be removed).

Dumping the optimized graph confirms the wrong attribute:

input graph	optimized graph	correct?
Transpose(perm=[0,1]) → MatMul → Add	Gemm{transA:1}	no
Transpose(perm=[1,0]) → MatMul → Add	Gemm{transA:1}	yes
MatMul → Add	Gemm{transA:0}	yes
Transpose(perm=[0,1]) → MatMul (no Add)	FusedMatMul{transA:0}	yes

The no-Add path (MatMulTransposeFusion → FusedMatMul) handles the identity
perm correctly, so only the Gemm path is affected.

Expected behavior

Both optimization levels return x @ y + b.

Scope confirmed

• Wrong on the A side and on the B side of the MatMul.
• Wrong for float32 and float16; float64 unaffected.
• Deterministic: 5/5 identical repeats.
• CPUExecutionProvider and CoreMLExecutionProvider both affected.
• Shapes checked: (3,3,3) (4,4,4) (5,5,5) (4,4,2) (8,8,3) silent wrong result;
  (2,3,2) (3,4,5) execution error.

To reproduce

import numpy as np, onnx, onnxruntime as ort
from onnx import helper, TensorProto

F = TensorProto.FLOAT
M = K = N = 3
nodes = [helper.make_node("Transpose", ["x"], ["t"], perm=[0, 1]),   # identity
         helper.make_node("MatMul", ["t", "y"], ["mm"]),
         helper.make_node("Add", ["mm", "b"], ["o"])]
g = helper.make_graph(nodes, "g",
    [helper.make_tensor_value_info("x", F, [M, K]),
     helper.make_tensor_value_info("y", F, [K, N]),
     helper.make_tensor_value_info("b", F, [M, N])],
    [helper.make_tensor_value_info("o", F, [M, N])])
m = helper.make_model(g, opset_imports=[helper.make_opsetid("", 21)], ir_version=10)
onnx.checker.check_model(m, full_check=True)

rng = np.random.default_rng(0)
feed = {"x": rng.standard_normal((M, K)).astype("float32"),
        "y": rng.standard_normal((K, N)).astype("float32"),
        "b": rng.standard_normal((M, N)).astype("float32")}
expected = feed["x"] @ feed["y"] + feed["b"]

for lvl in (ort.GraphOptimizationLevel.ORT_DISABLE_ALL,
            ort.GraphOptimizationLevel.ORT_ENABLE_ALL):
    so = ort.SessionOptions(); so.graph_optimization_level = lvl
    got = ort.InferenceSession(m.SerializeToString(), so,
                               providers=["CPUExecutionProvider"]).run(None, feed)[0]
    print(lvl, "match =", np.allclose(got, expected, atol=1e-5))

Output on 1.27.0:

GraphOptimizationLevel.ORT_DISABLE_ALL match = True
GraphOptimizationLevel.ORT_ENABLE_ALL  match = False

Using M = 2, K = 3, N = 2 turns the silent mismatch into
InvalidArgument ... Gemm: Invalid bias shape for broadcast.

Urgency

Medium-high. Identity transposes are routinely emitted by ONNX exporters and by
graph-rewriting tools, and the square-shape case corrupts inference results with
no error reported.

Platform

Mac

OS Version

macOS 26.0

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.27.0 (broken); 1.20.1 (correct)

ONNX Runtime API

Python

Architecture

ARM64

Execution Provider

Default CPU, CoreML

Execution Provider Library Version

N/A