onnxruntime DML EP cannot load some onnx files.

Describe the bug There is an RuntimeException error when I try to load an onnx file with DML EP. However, the CPU EP works fine for the same file. Error message: Microsoft.ML.OnnxRuntime.OnnxRuntimeException: [ErrorCode:RuntimeException] Exception during initialization:

Urgency There is no hard deadline, but this is important for me.

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows10
ONNX Runtime installed from (source or binary): binary
ONNX Runtime version: 1.12.1
DirectML version: 1.9.0
Visual Studio version (if applicable): 2019 (16.11.17)
GPU model: NVIDIA Geforce GTX 1650
Onnx model: Segmentation.onnx

To Reproduce The test c# code is quite simple

using Microsoft.ML.OnnxRuntime;

internal class Program
{
    static void Main(string[] args)
    {
        string modelFilePath = "Segmentation.onnx";
        var options = new SessionOptions();
        options.EnableMemoryPattern = false;
        options.ExecutionMode = ExecutionMode.ORT_SEQUENTIAL;
        options.LogSeverityLevel = OrtLoggingLevel.ORT_LOGGING_LEVEL_VERBOSE;
        options.GraphOptimizationLevel = GraphOptimizationLevel.ORT_DISABLE_ALL;
        options.AppendExecutionProvider_DML();
        var session = new InferenceSession(modelFilePath, options);
    }
}

And this is the result:

I have also tested at a c++ project, but there was no initialization error. Do you have any idea to deal with this?

Expected behavior Not to throw an exception.

Aug 10 '22 05:08 creaitr

I can not see exception message in the image. Would you, please, copy paste text?

Aug 11 '22 17:08 yuslepukhin

@yuslepukhin Yes, sure.

This is the entire log.

2022-08-12 12:52:57.4010505 [I:onnxruntime:, inference_session.cc:262 onnxruntime::InferenceSession::ConstructorCommon::<lambda_06048e3ee4d25a10314b7f89bee437e7>::operator ()] Flush-to-zero and denormal-as-zero are off
2022-08-12 12:52:57.4154053 [I:onnxruntime:, inference_session.cc:270 onnxruntime::InferenceSession::ConstructorCommon] Creating and using per session threadpools since use_per_session_threads_ is true
2022-08-12 12:52:57.4276310 [I:onnxruntime:, inference_session.cc:291 onnxruntime::InferenceSession::ConstructorCommon] Dynamic block base set to 0
2022-08-12 12:52:57.4558688 [I:onnxruntime:, inference_session.cc:1246 onnxruntime::InferenceSession::Initialize] Initializing session.
2022-08-12 12:52:57.4633939 [I:onnxruntime:, inference_session.cc:1283 onnxruntime::InferenceSession::Initialize] Adding default CPU execution provider.
2022-08-12 12:52:57.4826688 [V:onnxruntime:, session_state.cc:1186 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Node placements
2022-08-12 12:52:57.4908581 [V:onnxruntime:, session_state.cc:1188 onnxruntime::VerifyEachNodeIsAssignedToAnEp] All nodes have been placed on [DmlExecutionProvider].
2022-08-12 12:52:57.5011416 [V:onnxruntime:, session_state.cc:81 onnxruntime::SessionState::CreateGraphInfo] SaveMLValueNameIndexMapping
2022-08-12 12:52:57.5083453 [V:onnxruntime:, session_state.cc:127 onnxruntime::SessionState::CreateGraphInfo] Done saving OrtValue mappings.
2022-08-12 12:52:57.5173129 [I:onnxruntime:, session_state_utils.cc:140 onnxruntime::session_state_utils::SaveInitializedTensors] Saving initialized tensors.
2022-08-12 12:52:57.5261069 [I:onnxruntime:, session_state_utils.cc:268 onnxruntime::session_state_utils::SaveInitializedTensors] Done saving initialized tensors
2022-08-12 12:52:57.5465870 [E:onnxruntime:, inference_session.cc:1530 onnxruntime::InferenceSession::Initialize::<lambda_723fecf95a6df95ec35cfee1c1cc414b>::operator ()] Exception during initialization:

처리되지 않은 예외: Microsoft.ML.OnnxRuntime.OnnxRuntimeException: [ErrorCode:RuntimeException] Exception during initialization:
   위치: Microsoft.ML.OnnxRuntime.NativeApiStatus.VerifySuccess(IntPtr nativeStatus) 파일 D:\a\_work\1\s\csharp\src\Microsoft.ML.OnnxRuntime\NativeApiStatus.shared.cs:줄 31
   위치: Microsoft.ML.OnnxRuntime.InferenceSession.Init(String modelPath, SessionOptions options, PrePackedWeightsContainer prepackedWeightsContainer) 파일 D:\a\_work\1\s\csharp\src\Microsoft.ML.OnnxRuntime\InferenceSession.shared.cs:줄 776
   위치: Microsoft.ML.OnnxRuntime.InferenceSession..ctor(String modelPath, SessionOptions options) 파일 D:\a\_work\1\s\csharp\src\Microsoft.ML.OnnxRuntime\InferenceSession.shared.cs:줄 85
   위치: dotnetOnnxRuntimeTest.Program.Main(String[] args) 파일 C:\Users\jinbae\workspace\dotnetOnnxRuntimeTest\Program.cs:줄 33
계속하려면 아무 키나 누르십시오 . . .

Aug 12 '22 03:08 creaitr

This is being investigated

Aug 19 '22 17:08 yuslepukhin

oh， I meet the same problem! 2022-09-05 14:46:03.2726621 [E:onnxruntime:, inference_session.cc:1530 onnxruntime::InferenceSession::Initialize::<lambda_300271ae079a93faf30b4ef2a83d2ebc>::operator ()] Exception during initialization:

Sep 05 '22 07:09 dianxin556

I have applied the solution that was discussed at #13429 and your model loads successfully with DML.

f0c7d6a3>::operator ()] Flush-to-zero and denormal-as-zero are off 2022-10-27 13:59:23.6723378 [I:onnxruntime:, inference_session.cc:271 onnxruntime::InferenceSession::ConstructorCommon] Creating and using per session threadpools since use_per_session_threads_ is true 2022-10-27 13:59:23.6843943 [I:onnxruntime:, inference_session.cc:292 onnxruntime::InferenceSession::ConstructorCommon] Dynamic block base set to 0 2022-10-27 13:59:23.7919622 [I:onnxruntime:, inference_session.cc:1222 onnxruntime::InferenceSession::Initialize] Initializing session. 2022-10-27 13:59:23.7992591 [I:onnxruntime:, inference_session.cc:1259 onnxruntime::InferenceSession::Initialize] Adding default CPU execution provider. 2022-10-27 13:59:23.8225917 [V:onnxruntime:, session_state.cc:1010 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Node placements 2022-10-27 13:59:23.8308038 [V:onnxruntime:, session_state.cc:1013 onnxruntime::VerifyEachNodeIsAssignedToAnEp] All nodes placed on [DmlExecutionProvider]. Number of nodes: 1 2022-10-27 13:59:23.8413136 [V:onnxruntime:, session_state.cc:66 onnxruntime::SessionState::CreateGraphInfo] SaveMLValueNameIndexMapping 2022-10-27 13:59:23.8502356 [V:onnxruntime:, session_state.cc:112 onnxruntime::SessionState::CreateGraphInfo] Done saving OrtValue mappings. 2022-10-27 13:59:23.8601391 [I:onnxruntime:, session_state_utils.cc:199 onnxruntime::session_state_utils::SaveInitializedTensors] Saving initialized tensors. 2022-10-27 13:59:23.8681469 [I:onnxruntime:, session_state_utils.cc:342 onnxruntime::session_state_utils::SaveInitializedTensors] Done saving initialized tensors 2022-10-27 13:59:28.8467759 [I:onnxruntime:, inference_session.cc:1488 onnxruntime::InferenceSession::Initialize] Session successfully initialized.

Oct 27 '22 21:10 yuslepukhin