Issues with installing/running CUDA back-end

[jacobjelen]

Hi, I've been stuck on this for a while [noob] I have installed CUDA from Nvidia website. I'm trying to use CUDArray to run your neural_artistic_style algorithm using my GPU. On the CPU it works fine (super slow, but works)

At the CUDArray installation I set the environment variables, but when I do make i get this message

nvcc -gencode arch=compute_20,code=sm_20 -gencode arch=compute_20,code=compute_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_30,code=compute_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_35,code=compute_35 -O3 --compiler-options '-DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors' --ftz=true --prec-div=false -prec-sqrt=false --fmad=true -I./include -I/usr/local/cuda/include -c -o src/nnet/pool_b01.o src/nnet/pool_b01.cu make: nvcc: No such file or directory make: *** [src/nnet/pool_b01.o] Error 1

I have found the nvcc file at usr/local/cuda/bin/. I've also tried LD_LIBRARY_PATH='/usr/local/cuda/bin/' before doing make but that didn't help either.

Any tips? What am I doing wrong?

Thanks a lot!

[andersbll]

Hi there! Try export CUDA_PREFIX=/usr/local/cuda before make.

[jacobjelen]

Hi Anders, Thanks for getting back to me so quickly. I've tried that before but it's still giving me the same error. If I moved/copied the nvcc file somewhere it would find it, could that be a solution?

[andersbll]

Maybe you could try to inject som echo statements in the makefile to check the that environment variables are correct.

Are you running make with sudo? In that case you need to transfer you environment variables with sudo -E make.

[jacobjelen]

I'm not using sudo as I'm on an admin account.

I've tried replacing NVCC = nvcc with NVCC = /usr/local/cuda/bin/nvcc in the Makefile. I'm not sure if it's related but I got this error about missing * cudnn.h* library:

> g++ -DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors -I./include -I/usr/local/cuda/include -c -o src/nnet/cudnn.o src/nnet/cudnn.cpp
> In file included from src/nnet/cudnn.cpp:5:
> ./include/cudarray/nnet/cudnn.hpp:8:10: fatal error: 'cudnn.h' file not found
> #include <cudnn.h>
>          ^
> 1 error generated.
> make: *** [src/nnet/cudnn.o] Error 1

I found out that I need to register with NVIDIA to download the missing library and put it there manually. Found it here: https://github.com/BVLC/caffe/issues/1325

I did that and the error I'm getting now is this

g++ -DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors -I./include -I/usr/local/cuda/include -c -o src/nnet/cudnn.o src/nnet/cudnn.cpp
src/nnet/cudnn.cpp:74:17: fatal error: no matching function for call to 'cudnnSetPoolingNdDescriptor'
    CUDNN_CHECK(cudnnSetPoolingNdDescriptor(
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~
./include/cudarray/nnet/cudnn.hpp:87:44: note: expanded from macro 'CUDNN_CHECK'
#define CUDNN_CHECK(status) { cudnn_check((status), __FILE__, __LINE__); }
                                           ^
src/nnet/cudnn.cpp:111:16: note: in instantiation of member function 'cudarray::PoolBC01CuDNN<float>::fprop' requested here
template class PoolBC01CuDNN<float>;
               ^
/usr/local/cuda/include/cudnn.h:821:27: note: candidate function not viable: requires 7 arguments, but 6 were provided
cudnnStatus_t CUDNNWINAPI cudnnSetPoolingNdDescriptor(
                          ^
1 error generated.
make: *** [src/nnet/cudnn.o] Error 1

[andersbll]

Great job getting it working, though I'm still uncertain why setting CUDA_PREFIX doesn't help. This should ensure that PATH is correct in the Makefile.

Anyway, your last problem is probably caused by a mismatching version of cuDNN. I think you have installed cuDNN v5 RC. If so, you should checkout the cudnn5 branch of CUDArray. When v5 is no longer a release candidate, the cudnn5 branch will be merged to master.

[jacobjelen]

Ok, I've done that. It seems to have installed ok. But the Neural Artistic Style still won't run on my GPU. Still says CUDArray: CUDA back-end not available, using NumPy. I need to go now, but I'll look into it some more later and will post my results here. For now, thanks a lot for your help and work!

[andersbll]

My pleasure :)

For the last problem, make sure your LD_LIBRARY_PATH points to libcudarray.so and to the cuDNN libraries.

[00fq00]

Hello,have you solved this problem? @jacobjelen could you tell me how do you solve the last problem?