openvino icon indicating copy to clipboard operation
openvino copied to clipboard

[Bug]: [GPU] ProgramBuilder build failed on 2025.2, was inferring fine for years until 2024.6 included

Open JulienMaille opened this issue 6 months ago • 2 comments

OpenVINO Version

2025.2.0

Operating System

Windows System

Device used for inference

GPU

Framework

ONNX

Model used

UnetResnet18 see zip files below

Issue description

Since the update from 2024.6 to 2025.2 the model won't load or run on a Surface7 Pro when using OpenVino backend. It works with DmlExecutionProvider on the same GPU and also with OpenCV TARGET_OPENCL

Windows version: 10.0 (build 19045)
CPU: Intel(R) Core(TM) i5-7300U CPU @ 2.60GHz
OpenCV version: 4.12.0-dev
OpenVINO Intel GPU Adapter 0: Intel(R) HD Graphics 620
ONNX Runtime version: 1.22.0
DML Adapter 0: Intel(R) HD Graphics 620

Tested on other Intel GPU it works fine (like before) Intel(R) Arc(TM) A770M Graphics Intel(R) Iris(R) Xe Graphics

It fails both when using ONNXRuntime 1.22 or OpenCV 4.12.0-dev

 == Running on ONNXRuntime OpenVINOExecutionProvider HETERO:GPU,CPU ==
ONNX error: Exception during initialization: D:\Dev\onnxruntime\ort-1.22.0\onnxruntime\core\providers\openvino\ov_interface.cc:98 class onnxruntime::openvino_ep::OVExeNetwork __cdecl onnxruntime::openvino_ep::OVCore::CompileModel(class std::shared_ptr<class ov::Model const > &,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > &,class std::map<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class ov::Any,struct std::less<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > >,class std::allocator<struct std::pair<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const ,class ov::Any> > > &,const class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > &) [OpenVINO-EP]  Exception while Loading Network for graph: OpenVINOExecutionProvider_OpenVINO-EP-subgraph_1_0Exception from src\inference\src\cpp\core.cpp:112:
Exception from src\inference\src\dev\plugin.cpp:53:
Exception from src\plugins\hetero\src\compiled_model.cpp:33:
Standard exception from compilation library: Exception from src\inference\src\dev\plugin.cpp:53:
Check 'false' failed at src\plugins\intel_gpu\src\plugin\program_builder.cpp:163:
[GPU] ProgramBuilder build failed!
Program build failed(0_part_9):

Model './resnet18_fixed_dyn.onnx' load failed.
 == Running on OpenCV BACKEND_OPENVINO TARGET_OPENCL ==
1 detected device(s)
name: Intel(R) HD Graphics 620
Loading model..done in 35ms
OpenCV exception caught: OpenCV(4.12.0-dev) D:\Dev\opencv\modules\dnn\src\ie_ngraph.cpp:374: error: (-2:Unspecified error) in function 'cv::dnn::InfEngineNgraphNet::initPlugin'
> Failed to initialize Inference Engine backend (device = GPU): Exception from src\inference\src\cpp\core.cpp:112:
> Exception from src\inference\src\dev\plugin.cpp:53:
> Check 'false' failed at src\plugins\intel_gpu\src\plugin\program_builder.cpp:163:
> [GPU] ProgramBuilder build failed!
> Program build failed(0_part_0):
> 
> 
> 
> . Code: -2.

Step-by-step reproduction

Original model working with OpenCV dnn resnet18.zip Modified model with dynamic input for ONNXRuntime resnet18_fixed_dyn.zip

 session_options.SetGraphOptimizationLevel(ORT_ENABLE_ALL);

std::unordered_map<std::string, std::string> options;
options["device_type"] = "GPU"
options["precision"] = "FP32";
options["num_of_threads"] = "4";
options["disable_dynamic_shapes"] = params[3];

session_options.AppendExecutionProvider_OpenVINO_V2(options);

Relevant log output


Issue submission checklist

  • [x] I'm reporting an issue. It's not a question.
  • [x] I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
  • [x] There is reproducer code and related data files such as images, videos, models, etc.

JulienMaille avatar Jun 13 '25 18:06 JulienMaille

Hi @JulienMaille, Could you please share the installed GPU driver version? Also, could you please check if the issue can be reproduced using benchmark_app tool?

sshlyapn avatar Jun 16 '25 05:06 sshlyapn

@sshlyapn Sorry I should have checked first, I thought those drivers were auto-updated by Windows. I was running an old 27.20.100.8682 from 2020 (which was working fine until now) and after updating to 31.0.101.2135 it now works. Do you know why 2025.2.0 has this new requirement?

JulienMaille avatar Jun 17 '25 12:06 JulienMaille