Произошла ошибка в обработке клиента onnxruntime_providers_cuda.dll мой проект пробовал на GPU с библиотекой YoloV8.Gpu v4.0.0

[slaymurs]

Проверил Checking CUDA availability... CUDA is available! NVIDIA GeForce GTX 1080 Ti но проблема с библиотекой onnxruntime_providers_cuda.dll

Произошла ошибка в обработке клиента: [ErrorCode:RuntimeException] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1193 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\my\source\repos\test\smg\bin\Debug\net8.0\runtimes\win-x64\native\onnxruntime_providers_cuda.dll

На прошлой версии YoloV8 - 2.0.1 все работало как часы, но так как вышла новая версия YoloV8 - 4.0.0 переписана на .NET 8 я решил обновиться. На CPU работает отлично, а с GPU не хочет. Со своей стороны я уже все что знал перепробовал, не заводится. Может ошибка не у меня, а в самой библиотеке YoloV8.Gpu? Проверьте пожалуйста, очень жду.

[dme-compunet]

I don't think it's related to .NET 8, it makes more sense that it's related to the Microsoft.ML.OnnxRuntime.Gpu package. (try searching the official onnxruntime repo for similar problems)

Message ID: @.***>

[aloksharma1]

i think what you are missing is this https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html

[aloksharma1]

@dme-compunet on second i tried to test it on my system and i have right versions installed as per onnx guidelines, and still getting similar error

Microsoft.ML.OnnxRuntime.OnnxRuntimeException
  HResult=0x80131500
  Message=[ErrorCode:RuntimeException] D:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "G:\Dotnet\Desktop\Windows\ATTSystem\ATTSystem.Desktop\bin\Debug\net8.0\runtimes\win-x64\native\onnxruntime_providers_cuda.dll"

  Source=Microsoft.ML.OnnxRuntime
  StackTrace:
   at Microsoft.ML.OnnxRuntime.NativeApiStatus.VerifySuccess(IntPtr nativeStatus)
   at Microsoft.ML.OnnxRuntime.SessionOptions.AppendExecutionProvider_CUDA(Int32 deviceId)
   at Microsoft.ML.OnnxRuntime.SessionOptions.MakeSessionOptionWithCudaProvider(Int32 deviceId)
   at Compunet.YoloV8.YoloV8Builder.UseCuda(Int32 deviceId)
   at Compunet.YoloV8.YoloV8Builder.CreateDefaultBuilder()
   at Compunet.YoloV8.YoloV8Predictor.Create(BinarySelector model)
   at ATTSystem.Helpers.VehicleService.DetectVehicles(Mat frame) in G:\Dotnet\Desktop\Windows\ATTSystem\ATTSystem\Helpers\VehicleService.cs:line 28
   at ATTSystem.ViewModels.FeedViewModel.<StartVideoFeed>b__11_0() in G:\Dotnet\Desktop\Windows\ATTSystem\ATTSystem\ViewModels\FeedViewModel.cs:line 81
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)

PS C:\Users\ADMIN> nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Mar_28_02:30:10_Pacific_Daylight_Time_2024
Cuda compilation tools, release 12.4, V12.4.131
Build cuda_12.4.r12.4/compiler.34097967_0

PS C:\Users\ADMIN> nvidia-smi
Fri Apr 19 23:33:29 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 552.22                 Driver Version: 552.22         CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                     TCC/WDDM  | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4080 ...  WDDM  |   00000000:01:00.0  On |                  N/A |
|  0%   35C    P8             13W /  320W |    1439MiB /  16376MiB |      1%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      1712    C+G   ...siveControlPanel\SystemSettings.exe      N/A      |
|    0   N/A  N/A      2116    C+G   C:\Windows\System32\dwm.exe                 N/A      |
|    0   N/A  N/A      5868    C+G   ...U\ServiceHub.ThreadedWaitDialog.exe      N/A      |
|    0   N/A  N/A      7540    C+G   C:\Windows\explorer.exe                     N/A      |
|    0   N/A  N/A      8740    C+G   ...nt.CBS_cw5n1h2txyewy\SearchHost.exe      N/A      |
|    0   N/A  N/A      8772    C+G   ...2txyewy\StartMenuExperienceHost.exe      N/A      |
|    0   N/A  N/A     10008    C+G   ...US\ArmouryDevice\asus_framework.exe      N/A      |
|    0   N/A  N/A     10076    C+G   ...m Files\Mozilla Firefox\firefox.exe      N/A      |
|    0   N/A  N/A     10936    C+G   ...t.LockApp_cw5n1h2txyewy\LockApp.exe      N/A      |
|    0   N/A  N/A     11652    C+G   ...inaries\Win64\EpicGamesLauncher.exe      N/A      |
|    0   N/A  N/A     11676    C+G   ...2\Enterprise\Common7\IDE\devenv.exe      N/A      |
|    0   N/A  N/A     11996    C+G   ...\Local\slack\app-4.37.101\slack.exe      N/A      |
|    0   N/A  N/A     12628    C+G   ...ekyb3d8bbwe\PhoneExperienceHost.exe      N/A      |
|    0   N/A  N/A     13632    C+G   ...5n1h2txyewy\ShellExperienceHost.exe      N/A      |
|    0   N/A  N/A     17364    C+G   ...CBS_cw5n1h2txyewy\TextInputHost.exe      N/A      |
|    0   N/A  N/A     17504    C+G   C:\Program Files\Listary\Listary.exe        N/A      |
|    0   N/A  N/A     17568    C+G   ...crosoft\Edge\Application\msedge.exe      N/A      |
|    0   N/A  N/A     21700    C+G   ...ne\Binaries\Win64\EpicWebHelper.exe      N/A      |
|    0   N/A  N/A     24596    C+G   C:\Windows\System32\Taskmgr.exe             N/A      |
|    0   N/A  N/A     24664    C+G   ...__8wekyb3d8bbwe\WindowsTerminal.exe      N/A      |
|    0   N/A  N/A     25416    C+G   ...n\NVIDIA App\CEF\NVIDIA Overlay.exe      N/A      |
|    0   N/A  N/A     26108    C+G   ...n\NVIDIA App\CEF\NVIDIA Overlay.exe      N/A      |
|    0   N/A  N/A     26324    C+G   ...on\123.0.2420.97\msedgewebview2.exe      N/A      |
|    0   N/A  N/A     27952    C+G   ...m Files\Mozilla Firefox\firefox.exe      N/A      |
|    0   N/A  N/A     28348    C+G   ...on\123.0.2420.97\msedgewebview2.exe      N/A      |
|    0   N/A  N/A     30716    C+G   ...\cef\cef.win7x64\steamwebhelper.exe      N/A      |
|    0   N/A  N/A     31396    C+G   ..._x64__kzf8qxf38zg5c\Skype\Skype.exe      N/A      |
|    0   N/A  N/A     31500    C+G   ..._x64__kzf8qxf38zg5c\Skype\Skype.exe      N/A      |
|    0   N/A  N/A     33040    C+G   ...2\Enterprise\Common7\IDE\devenv.exe      N/A      |
|    0   N/A  N/A     35108    C+G   ...wekyb3d8bbwe\XboxGameBarWidgets.exe      N/A      |
|    0   N/A  N/A     38000    C+G   C:\Windows\explorer.exe                     N/A      |
|    0   N/A  N/A     40044    C+G   ...on\123.0.2420.97\msedgewebview2.exe      N/A      |
|    0   N/A  N/A     42452    C+G   ...U\ServiceHub.ThreadedWaitDialog.exe      N/A      |
+-----------------------------------------------------------------------------------------+

i went to code & bumped onnx lib version and compile your code to see if this solves the issue, but the issue stays same.

[aloksharma1]

я думаю, что вам не хватает https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html

я же написал, что перепробовал всевозможные варианты, проблема в новых версиях библиотеки YoloV8.Gpu.dll v4.0.0 YoloV8.Gpu.dll v4.1.4 YoloV8.Gpu.dll v4.1.5 осталась не решенной. Мне пришлось вернутся к более старой версии библиотеки YoloV8.dll v3.1.1 для GPU, и там все работает. Проблема где то в новой версии самой библиотеки YoloV8.Gpu.dll.

check this https://onnxruntime.ai/docs/install/#install-onnx-runtime-gpu-cuda-12x-1, do you have cuda toolkit & cudnn installed on right version, i was having the same error but dowgrading cuda toolkit to 11.8 solved my issue but now i am facing another problem its still using cpu with gpu packages installed so i am trying to figure out the problem on onnx side, let me know if this worked for you? you need to match cuda version compatible with your onnx runtime version.

@dme-compunet can you help, even when rebuilding your project for gpu its not using gpu when running the program? i solved this issue in debugging by using #if GPURELEASE || DEBUG but i am sure when running my main project where i am using this lib is not hitting gpu initializations. so thats the issue on your part (if i solve it i will make a pr).

useful notes: https://forums.developer.nvidia.com/t/could-not-load-library-cudnn-cnn-infer64-8-dll-error-code-193/218437/16

[Fwdchina]

Hello, In your example code, using YOLOv8 for pose estimation allows you to obtain the bounding box size, confidence level, and processing speed. Could you please advise on how to obtain coordinate information for key points? I appreciate your assistance and eagerly await your response.

[dme-compunet]

@IWillAshore The method Pose returns PoseResult object which contains the property Boxes as an array of PoseBoundingBox that they contain the property: Keypoints, Obviously?

[Fwdchina]

var imagePath = "MyImagePath"; var predictor = YoloV8Predictor.Create("MyONNXPath"); var result = await predictor.PoseAsync(imagePath); and I've found that it only includes the BOX property： Could you please provide an example of code that returns a PoseResult object?

[Fwdchina]

I have obtained the coordinates. Thank you very much!

[dme-compunet]

I'm sorry, believe me if I had a solution to your problem I would write it here, but I don't use GPU at all, I added a special nuget package for people who want to use GPU, assuming they will have to deal with problems on their own (or with help of the community) because I have no experience with it at all, anyway I'm reopening the issue.