Error running on GPU (DML)

[00f8t]

I`m trying to run it on GPU using DML. I installed required packages

and when im trying to set provider

var sessionOptions = new SessionOptions()
{
    LogSeverityLevel = OrtLoggingLevel.ORT_LOGGING_LEVEL_VERBOSE,
    LogVerbosityLevel = 1,
    EnableMemoryPattern = false
};
sessionOptions.AppendExecutionProvider_DML();

it breaks with next error:
Microsoft.ML.OnnxRuntime.OnnxRuntimeException: '[ErrorCode:RuntimeException] '

Also i tried running it on CUDA, but it was even slower, than on CPU, and i tried to run it using TensorRT, but there was problems with model, i guess..

Im running on RTX 3050, and DX12 is installed.

[00f8t]

microsoft/onnxruntime#13429 (comment)

this helped, but it still runs slower, than on CPU..

[georg-jung]

You could try the non-quantized model instead: https://www.nuget.org/packages/FaceAiSharp.Models.ArcFace.LResNet100E-IR

And specifiy the model path manually, e.g.

var modelPath = Path.Combine(GetExeDir(), "onnx", "arcfaceresnet100-11.onnx");
var opt = new ArcFaceEmbeddingsGeneratorOptions() { ModelPath = modelPath };
return new ArcFaceEmbeddingsGenerator(opt, sessionOptions);

I think I remember cases where the non-quantized version was actually faster, but tbh my experience with different inference setups is limited.