onnxruntime
onnxruntime copied to clipboard
[Documentation] Setup the CUDA Environment is not detailed enough
Describe the documentation issue
I'm following this guide, which links to this page.
The instructions on that latter page are not detailed enough. I have "CUDA_PATH" set correctly, and it's working for other things that require CUDA 11.8.
But I'm still getting DLL LOAD ERRORS when I try to run the script -- but it's an ONNX DLL and not a CUDA one. And the DLL appears to actually exist. So, it's not really clear what the issue is to me.
onnxruntime_genai.onnxruntime_genai.OrtException: C:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1426 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\Mikey\miniconda3\envs\phi3\Lib\site-packages\onnxruntime_genai\onnxruntime_providers_cuda.dll"
Proof that the DLL Exists:
> dir C:\Users\Mikey\miniconda3\envs\phi3\Lib\site-packages\onnxruntime_genai\onnxruntime_providers_cuda.dll
Volume in drive C is Windows
Volume Serial Number is 784B-4CD0
Directory of C:\Users\Mikey\miniconda3\envs\phi3\Lib\site-packages\onnxruntime_genai
06/27/2024 11:53 AM 410,648,608 onnxruntime_providers_cuda.dll
1 File(s) 410,648,608 bytes
0 Dir(s) 1,124,985,217,024 bytes free
Also proof of CUDA_PATH being set:
> dir "%CUDA_PATH%"
Volume in drive C is Windows
Volume Serial Number is 784B-4CD0
Directory of C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
10/20/2023 02:08 PM <DIR> .
10/20/2023 02:06 PM <DIR> ..
10/20/2023 02:08 PM <DIR> bin
10/20/2023 02:08 PM <DIR> compute-sanitizer
08/25/2022 03:33 PM 80,660 CUDA_Toolkit_Release_Notes.txt
08/25/2022 03:33 PM 160 DOCS
08/25/2022 03:33 PM 61,498 EULA.txt
10/20/2023 02:08 PM <DIR> extras
10/20/2023 02:08 PM <DIR> include
10/20/2023 02:08 PM <DIR> lib
10/20/2023 02:08 PM <DIR> libnvvp
10/20/2023 02:08 PM <DIR> nvml
10/20/2023 02:08 PM <DIR> nvvm
08/25/2022 03:33 PM 524 README
10/20/2023 02:08 PM <DIR> src
10/20/2023 02:08 PM <DIR> tools
09/29/2022 11:17 AM 2,608 version.json
5 File(s) 145,450 bytes
12 Dir(s) 1,124,984,737,792 bytes free
Page / URL
https://onnxruntime.ai/docs/genai/howto/setup-cuda-env.html#special-instructions-on-windows