warp icon indicating copy to clipboard operation
warp copied to clipboard

Cannot detect GPU

Open tnnandi opened this issue 1 year ago • 2 comments

I have installed warp within a conda environment on WSL2 on Windows (on a machine with GeForce RTX 3050 GPU), but warp doesn't seem to be able to locate the GPU.

Here are some relevant information and output:

nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Mar_28_02:18:24_PDT_2024
Cuda compilation tools, release 12.4, V12.4.131
Build cuda_12.4.r12.4/compiler.34097967_0
 python
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import warp as wp
>>> wp.init()
Warp CUDA error 100: no CUDA-capable device is detected (in function init_cuda_driver, /builds/omniverse/warp/warp/native/cuda_util.cpp:224)
Warp CUDA error 3: initialization error (in function cuda_device_get_count, /builds/omniverse/warp/warp/native/warp.cu:1635)
Warp 1.1.0 initialized:
   CUDA devices not available
   Devices:
     "cpu"      : "x86_64"
   Kernel cache:
     /home/tnandi/.cache/warp/1.1.0

tnnandi avatar May 29 '24 22:05 tnnandi

same

maohaotian avatar May 30 '24 17:05 maohaotian

Thanks for reporting this! CUDA support on WSL can be a bit tricky to set up. Did you follow the steps at CUDA on WSL? Pay specific attention to:

users must not install any NVIDIA GPU Linux driver within WSL 2. One has to be very careful here as the default CUDA Toolkit comes packaged with a driver, and it is easy to overwrite the WSL 2 NVIDIA driver with the default installation. We recommend developers to use a separate CUDA Toolkit for WSL 2 (Ubuntu) available from the CUDA Toolkit Downloads page to avoid this overwriting.

The above page also contains other troubleshooting advice. Please let us know if you're able to run other CUDA workloads within WSL, but not Warp.

c0d1f1ed avatar May 30 '24 17:05 c0d1f1ed

Hi,

After the correct installation of CUDA and Warp, I still can't detect the CUDA device in warp in WSL2. How can I fix it? Thx.

My env is Python=3.10, warp-lang==1.3.0+cu11, CUDA=11.8

Error: Warp CUDA error 100: no CUDA-capable device is detected (in function init_cuda_driver, /builds/omniverse/warp/warp/native/cuda_util.cpp:232) Warp CUDA error 3: initialization error (in function cuda_device_get_count, /builds/omniverse/warp/warp/native/warp.cu:1636) Warp 1.3.0 initialized: CUDA devices not available Devices: "cpu" : "x86_64" Kernel cache: /root/.cache/warp/1.3.0 Available devices: ['cpu']

jamesdemon923 avatar Jul 28 '24 06:07 jamesdemon923

Hi!

I've just tried installing a fresh WSL2, then I ran sudo apt update && sudo apt upgrade to pull the latest packages for everything and, finally, I installed Warp using pip install warp-lang (I did that using venv, but it doesn't matter).

With only these steps, Warp was able to detect the GPU upon calling wp.init():

(.venv) christopher@christopher-workstation:~$ python3
Python 3.10.12 (main, Mar 22 2024, 16:50:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import warp as wp
>>> wp.init()
Warp 1.3.1 initialized:
   CUDA Toolkit 12.5, Driver 12.2
   Devices:
     "cpu"      : "x86_64"
     "cuda:0"   : "NVIDIA GeForce RTX 3090 Ti" (24 GiB, sm_86, mempool enabled)
   Kernel cache:
     /home/christopher/.cache/warp/1.3.1

Do you have any hint on what you may have done differently that might help pinpointing/reproducing the issue? Would you able by any chance to reinstall WSL2, follow the same steps, and report on the result?

christophercrouzet avatar Jul 29 '24 02:07 christophercrouzet

Also, a good suggestion from @shi-eric: are you able to run the cuda-samples successfully?

christophercrouzet avatar Jul 29 '24 02:07 christophercrouzet

Thank you very much! Reinstalling WSL2 solved my problem!

jamesdemon923 avatar Jul 29 '24 03:07 jamesdemon923

I am having the same issue, but I do not want to redownload WSL as I have many months of work done on my WSL. My nvidia-smi works and I can access my GPU through CUDA with everything else (such as pytorch) no problem except when using warp. Is there anything I can do like manually remove the driver from WSL or

Warp CUDA error 100: no CUDA-capable device is detected (in function init_cuda_driver, /builds/omniverse/warp/warp/native/cuda_util.cpp:240) Warp CUDA error 3: initialization error (in function cuda_device_get_count, /builds/omniverse/warp/warp/native/warp.cu:1685) Warp 1.5.0 initialized: CUDA devices not available Devices: "cpu" : "x86_64" Kernel cache: /home/joe/.cache/warp/1.5.0

Okay I figured it out: I ran: dpkg -l | grep -i driver and dpkg -l | grep -i nvidia And saw that I had somehow downloaded this linux driver so I removed it and restarted WSL2 and warp was running fine: sudo apt purge nvidia-driver-* libnvidia-*

JoeDoerr avatar Dec 13 '24 19:12 JoeDoerr

For zsh:

sudo apt purge "nvidia-driver-*" "libnvidia-*"

(just in case)

camille-dupont avatar Mar 26 '25 14:03 camille-dupont