How to check which CPU ISA (AVX/AVX512/AMX) was used in inference

[TuongNguyen94]

Hi all,
I am doing a project which aims to run a model on Intel's AMX. I quantized the model, built Onnxruntime with --use_dnnl flag then deployed the built library. However, when I run inference in C++, I cannot find a way to verify that the model uses AMX. Is there any way to find this out? I have tried Ort logs but seen nothing beside "provider" : "CPUExecutionProvider"

[Robert Knight (robertknight)]

Is there any way to find this out?

One method to find out is to run model inference under a profiler and inspect the functions where most time is spent (assuming the model spends most of its time in MatMul, Conv or Gemm operations). You can then usually tell from the name which one is being used, or if you go into the assembly view, see which instructions are being used. samply is a profiler that works on Linux / Mac / Windows.

[TuongNguyen94]

Thank you Robert.