neon icon indicating copy to clipboard operation
neon copied to clipboard

Invalid choice: "gpu"

Open Pavel-Konarik opened this issue 8 years ago • 12 comments

Hi All, I have a problem running neon with GPU, every script is running fine, but only option is CPU. While trying to solve this issues I have :

  1. Installed a clean ubuntu 16.04.01
  2. Installed Nvidia driver 361.42
  3. Installed Cuda 7.5 toolkit (examples run totally fine)
  4. Exported Cuda paths
  5. Installed neon

I have tried adding -b gpu while running examples, but it ended with error : error: argument -b/--backend: invalid choice: 'gpu' (choose from 'cpu') Aka, the only option is CPU

Any ideas what I did wrong?

Pavel-Konarik avatar Sep 24 '16 14:09 Pavel-Konarik

Can you try this command from the neon directory and tell us what it prints?

nvcc neon/backends/util/check_gpu.c && ./a.out; echo $?

anlthms avatar Sep 27 '16 14:09 anlthms

Thanks for the reply. The output is 0. (.venv2) root@a:/home/ai/ai/neon# nvcc neon/backends/util/check_gpu.c && ./a.out; echo $? 0

Pavel-Konarik avatar Sep 27 '16 15:09 Pavel-Konarik

Does this command return a number >= 3.0? If not, you're out of luck :-(

./neon/backends/util/check_gpu.py

Note that you should activate the neon virtual env for this command to work.

anlthms avatar Sep 27 '16 18:09 anlthms

Thanks for the help (.venv2) root@a:/home/ai/ai/neon# ./neon/backends/util/check_gpu.py DISPLAY:neon:0 (.venv2) root@a:/home/ai/ai/neon# I have no idea what went wrong. Does the Cuda toolkit need to be installed before the neon make? I am quite sure I had it correctly set up beforehand, but I am out of ideas.

Pavel-Konarik avatar Sep 27 '16 18:09 Pavel-Konarik

It looks like you don't have a good pycuda installation.

anlthms avatar Sep 27 '16 19:09 anlthms

I can confirm that building pycuda from source (https://github.com/inducer/pycuda) solved the problem for me. Configuration CUDA 8 / nvidia driver 370.28 / ArchLinux

bitstormFA avatar Oct 06 '16 17:10 bitstormFA

I'm in the same position, but when i do: ./neon/backends/util/check_gpu.py i get 0

What does this mean? My gpu is not suitable for Neon?

klezmen avatar Oct 14 '16 23:10 klezmen

Most likely you have an issue with pycuda. Trying this in the python interpreter might provide more clues:

import pycuda.driver as drv

anlthms avatar Oct 18 '16 14:10 anlthms

Hi I had this same exact issue. However, I haven't been able to resolve this one and I have been able to resolve all my issues so far.

GTX 1080 driver 367.48 Ubuntu 16.04 Can switch between Cuda Toolkit 7.5 and 8.0 via symlinks to /usr/local/cuda

I built pycuda from source. The tgz copy from his website seems to looks for libcurand.so.8.0 as I had that library not found after doing ldd _driver.so. I then cloned from github and this copy looked for 7.5 and saw that I had it (symlinked to 7.5 at that time). I even got import pycuda.driver as drv when symlinked to 7.5. That said, I then get "OSError: CUDA runtime library not found".

If I symlink to the cuda-8.0 directory, your (@anlthms) covnet runs, even though the backend defaults to cpu and the pycuda driver import complains about not having libcurand.so.7.5.

@anlthms Do you (or anyone else reading) have any leads on this? Do you only run 7.5?

Thanks for all the help so far!

Cpruce avatar Nov 02 '16 03:11 Cpruce

@Cpruce try this

had the same issue, but fixed it with a much easier solution than recompiling pycuda.

check_gpu.py was returning DISPLAY:neon:0.0

when I ran the import suggestion that @anlthms posted, I got an ImportError for libcurand.so.8.0 I fixed it by setting the LD_LIBRARY_PATH environment variable with export on the command line like this:

export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64/

now check_gpu.py returns DISPLAY:neon:5.0

you have to add the export command to the .bashrc file in your home directory if you want to make the export permanent

Clayton-Davis avatar Dec 10 '16 20:12 Clayton-Davis

I have the same issue: error: argument -b/--backend: invalid choice: 'gpu' (choose from 'cpu')

nvcc neon/backends/util/check_gpu.c && ./a.out; echo $? returns: 0

./neon/backends/util/check_gpu.py returns: DISPLAY:neon:3.7

All this after iinstalling pycuda from source... Cuda 8.0 toolset LD_LIBRARY_PATH=LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64

I am running on a Google Compute Engine instance w/ Nvidia Tesla K80 OS = Ubuntu 16.04

Any other ideas on what might be wrong here? Thanks

sbass301 avatar Apr 21 '17 04:04 sbass301

Device 0 does not have CUDA compute capability 3.0 or greater Is there any way to use neon with 2.1 capability GPU?

iNomaD avatar Oct 18 '17 17:10 iNomaD