[Documentation] Setup the CUDA Environment is not detailed enough

[BrainSlugs83]

Describe the documentation issue

I'm following this guide, which links to this page.

The instructions on that latter page are not detailed enough. I have "CUDA_PATH" set correctly, and it's working for other things that require CUDA 11.8.

But I'm still getting DLL LOAD ERRORS when I try to run the script -- but it's an ONNX DLL and not a CUDA one. And the DLL appears to actually exist. So, it's not really clear what the issue is to me.

onnxruntime_genai.onnxruntime_genai.OrtException: C:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1426 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\Mikey\miniconda3\envs\phi3\Lib\site-packages\onnxruntime_genai\onnxruntime_providers_cuda.dll"

Proof that the DLL Exists:

> dir C:\Users\Mikey\miniconda3\envs\phi3\Lib\site-packages\onnxruntime_genai\onnxruntime_providers_cuda.dll
 Volume in drive C is Windows
 Volume Serial Number is 784B-4CD0

 Directory of C:\Users\Mikey\miniconda3\envs\phi3\Lib\site-packages\onnxruntime_genai

06/27/2024  11:53 AM       410,648,608 onnxruntime_providers_cuda.dll
               1 File(s)    410,648,608 bytes
               0 Dir(s)  1,124,985,217,024 bytes free

Also proof of CUDA_PATH being set:

> dir "%CUDA_PATH%"
 Volume in drive C is Windows
 Volume Serial Number is 784B-4CD0

 Directory of C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8

10/20/2023  02:08 PM    <DIR>          .
10/20/2023  02:06 PM    <DIR>          ..
10/20/2023  02:08 PM    <DIR>          bin
10/20/2023  02:08 PM    <DIR>          compute-sanitizer
08/25/2022  03:33 PM            80,660 CUDA_Toolkit_Release_Notes.txt
08/25/2022  03:33 PM               160 DOCS
08/25/2022  03:33 PM            61,498 EULA.txt
10/20/2023  02:08 PM    <DIR>          extras
10/20/2023  02:08 PM    <DIR>          include
10/20/2023  02:08 PM    <DIR>          lib
10/20/2023  02:08 PM    <DIR>          libnvvp
10/20/2023  02:08 PM    <DIR>          nvml
10/20/2023  02:08 PM    <DIR>          nvvm
08/25/2022  03:33 PM               524 README
10/20/2023  02:08 PM    <DIR>          src
10/20/2023  02:08 PM    <DIR>          tools
09/29/2022  11:17 AM             2,608 version.json
               5 File(s)        145,450 bytes
              12 Dir(s)  1,124,984,737,792 bytes free

Page / URL

https://onnxruntime.ai/docs/genai/howto/setup-cuda-env.html#special-instructions-on-windows