caffe2 GPU support issue

[shaozi]

trafficstars

On ubuntu 17.10, installed cuda 8 and libcudnn 5.1 make and install coffe2 without issue. However, when running "python -m caffe2.python.operator_test.relu_op_test", GPU is not supported. I have NVIDIA 1080 installed.

Output:

WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
WARNING:root:Debug message: No module named caffe2_pybind11_state_gpu
...

[pietern]

Can you post your CMake summary output? This will include whether you compiled with CUDA or not.

[ccs1605]

`ray@ray-MS-7996:~$ find / -name caffe2_pybind11_state* 2>/dev/null /home/ray/caffe2/build/caffe2/python/caffe2_pybind11_state.so /home/ray/caffe2/build/caffe2/CMakeFiles/caffe2_pybind11_state.dir /home/ray/.PyCharm2017.3/system/python_stubs/-1943938901/caffe2/python/caffe2_pybind11_state.py /usr/local/caffe2/python/caffe2_pybind11_state.so /usr/local/caffe2/CMakeFiles/caffe2_pybind11_state.dir

ray@ray-MS-7996:~$ echo $PYTHONPATH /usr/local::/home/ray/caffe2/build ` i have the same issue. please advice!

[pjh5]

@ccs1605 @shaozi are you still encountering this problem?

If so, please post cmake output. It would seem that cmake is not finding your CUDA installations, and so is not building with GPU support

[ccs1605]

`CMake Warning at cmake/public/gflags.cmake:2 (find_package): By not providing "Findgflags.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "gflags", but CMake did not find one.

Could not find a package configuration file provided by "gflags" with any of the following names:

gflagsConfig.cmake
gflags-config.cmake

Add the installation prefix of "gflags" to CMAKE_PREFIX_PATH or set "gflags_DIR" to a directory containing one of the above files. If "gflags" provides a separate development package or SDK, be sure it has been installed. Call Stack (most recent call first): cmake/Dependencies.cmake:87 (include) CMakeLists.txt:88 (include)

-- Cannot find gflags with config files. Using legacy find. -- Found gflags (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so) CMake Warning at cmake/public/glog.cmake:2 (find_package): By not providing "Findglog.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "glog", but CMake did not find one.

Could not find a package configuration file provided by "glog" with any of the following names:

glogConfig.cmake
glog-config.cmake

Add the installation prefix of "glog" to CMAKE_PREFIX_PATH or set "glog_DIR" to a directory containing one of the above files. If "glog" provides a separate development package or SDK, be sure it has been installed. Call Stack (most recent call first): cmake/Dependencies.cmake:102 (include) CMakeLists.txt:88 (include)

-- Cannot find glog. Using legacy find. -- git Version: v0.0.0-dirty -- Version: 0.0.0 -- Performing Test HAVE_STD_REGEX -- Performing Test HAVE_STD_REGEX -- Performing Test HAVE_STD_REGEX -- success -- Performing Test HAVE_GNU_POSIX_REGEX -- Performing Test HAVE_GNU_POSIX_REGEX -- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile -- Performing Test HAVE_POSIX_REGEX -- Performing Test HAVE_POSIX_REGEX -- Performing Test HAVE_POSIX_REGEX -- success -- Performing Test HAVE_STEADY_CLOCK -- Performing Test HAVE_STEADY_CLOCK -- Performing Test HAVE_STEADY_CLOCK -- success -- Found lmdb (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so) -- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so) -- Found Snappy (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libsnappy.so) -- Could NOT find RocksDB (missing: RocksDB_INCLUDE_DIR RocksDB_LIBRARIES) CMake Warning at cmake/Dependencies.cmake:180 (message): Not compiling with RocksDB. Suppress this warning with -DUSE_ROCKSDB=OFF Call Stack (most recent call first): CMakeLists.txt:88 (include)

-- OpenCV found (/usr/share/OpenCV) CMake Warning at cmake/Dependencies.cmake:244 (find_package): By not providing "FindEigen3.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "Eigen3", but CMake did not find one.

Could not find a package configuration file provided by "Eigen3" with any of the following names:

Eigen3Config.cmake
eigen3-config.cmake

Add the installation prefix of "Eigen3" to CMAKE_PREFIX_PATH or set "Eigen3_DIR" to a directory containing one of the above files. If "Eigen3" provides a separate development package or SDK, be sure it has been installed. Call Stack (most recent call first): CMakeLists.txt:88 (include)

-- Did not find system Eigen. Using third party subdirectory. -- Found PythonInterp: /usr/bin/python (found suitable version "2.7.12", minimum required is "2.7") -- NumPy ver. 1.14.0 found (include: /usr/local/lib/python2.7/dist-packages/numpy/core/include) -- Could NOT find pybind11 (missing: pybind11_INCLUDE_DIR) CMake Error at /home/ray/caffe2/build/CMakeFiles/CMakeTmp/cmTC_61679Targets.cmake:16 (add_library): Target "MPI::MPI_C" links to item " -Wl,-rpath -Wl,/usr/lib/openmpi/lib -Wl,--enable-new-dtags" which has leading or trailing whitespace. This is now an error according to policy CMP0004. Call Stack (most recent call first): /home/ray/caffe2/build/CMakeFiles/CMakeTmp/CMakeLists.txt:12 (include)

CMake Error at /home/ray/caffe2/build/CMakeFiles/CMakeTmp/cmTC_61679Targets.cmake:16 (add_library): Target "MPI::MPI_C" links to item " -Wl,-rpath -Wl,/usr/lib/openmpi/lib -Wl,--enable-new-dtags" which has leading or trailing whitespace. This is now an error according to policy CMP0004. Call Stack (most recent call first): /home/ray/caffe2/build/CMakeFiles/CMakeTmp/CMakeLists.txt:12 (include)

CMake Error at /home/ray/cmake-3.10.2/Modules/FindMPI.cmake:970 (try_compile): Failed to generate test project build system. Call Stack (most recent call first): /home/ray/cmake-3.10.2/Modules/FindMPI.cmake:1011 (_MPI_try_staged_settings) /home/ray/cmake-3.10.2/Modules/FindMPI.cmake:1261 (_MPI_check_lang_works) cmake/Dependencies.cmake:279 (find_package) CMakeLists.txt:88 (include)

-- Configuring incomplete, errors occurred! See also "/home/ray/caffe2/build/CMakeFiles/CMakeOutput.log". See also "/home/ray/caffe2/build/CMakeFiles/CMakeError.log".

`

i checked another post and fixed by switching to cudnn5.1. but i dont think it is an issue here.

[pjh5]

@ccs1605 I think you should try disabling CMP0004, see https://cmake.org/cmake/help/v3.10/manual/cmake-policies.7.html on how to disable cmake policies. Please let us know if this causes any issues, as that policy https://cmake.org/cmake/help/v3.10/policy/CMP0004.html suggests that this may be indicative of an issue somewhere else.

What version of openmpi do you have? You can try reinstalling openmpi to or bumping to a recent version.

[khanhha]

after reading cmakelist.txt, I found out that Caffe is now a submodule, so we have to use the command " git submodule update --init --recursive" after pulling openpose.