[CMake][cuBLAS]Unable to build cublas backend with a default CMake command

[Rbiessy]

Summary

I am not able to build the cuBLAS backend without working around some CMake logic and adding CMake flags.

Version

I am using the latest commit on develop https://github.com/oneapi-src/oneMKL/commit/6aa5924d8d781ccba5fd7697f8b57b40b6c0d203.

Environment

I am using Ubuntu 22.04, and the compilers from oneAPI 2024.0.1. I am using module load compiler to set up my environment. I have tried with:

• CUDA toolkit 12.3.0 and CMake 3.26.3
• CUDA toolkit 11.8.0 and CMake 3.13.5

Steps to reproduce

$ cmake -Bbuild-blas-cuda -GNinja -DTARGET_DOMAINS=blas -DENABLE_MKLCPU_BACKEND=OFF -DENABLE_MKLGPU_BACKEND=OFF -DENABLE_CUBLAS_BACKEND=ON -DBUILD_FUNCTIONAL_TESTS=OFF .
-- CMAKE_BUILD_TYPE: None, set to Release by default
-- C compiler: icx was found in PATH, using icx
-- The CXX compiler identification is IntelLLVM 2024.0.2
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /path/to/oneapi/2024.0.1.46/compiler/2024.0/bin/compiler/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- TARGET_DOMAINS: blas
-- Looking for dpc++
-- Performing Test is_dpcpp
-- Performing Test is_dpcpp - Success
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Found CUDA: /path/to/cuda/12.3.0 (found suitable version "12.3", minimum required is "10.0") 
CMake Error at /path/to/cmake/3.26.3/share/cmake-3.26/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find cuBLAS (missing: OPENCL_INCLUDE_DIR)
Call Stack (most recent call first):
  /path/to/cmake/3.26.3/share/cmake-3.26/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
  cmake/FindcuBLAS.cmake:54 (find_package_handle_standard_args)
  src/blas/backends/cublas/CMakeLists.txt:22 (find_package)

Observed behavior

The issue is that cmake/FindcuBLAS.cmake requires some variables which are not set anymore. In general and from my previous experience I don't think the files cmake/FindcuBLAS.cmake and similar are needed anymore. It would be simpler to use a solution similar to how cuFFT is linked here.

I can workaround that by removing the required variables OPENCL_INCLUDE_DIR and CUDA_CUDA_LIBRARY. I am then running into a different issue where the CMake logic chooses to use clang++ by default instead of icpx and is not setting CXX_STANDARD 17 because of this. I believe the recommendation is to use icpx so shouldn't CMake select this compiler by default if it is found in the PATH, like it does for MKLCPU or MKLGPU backends?

To work around this, I first need to add /path/to/intel/oneapi/2024.0.1.46/compiler/2024.0/bin/compiler to PATH so that oneAPI clang++ is used instead of system clang++. The module script does not seem to add the path to clang++ anymore since a recent release. The compilation still fails because -std=gnu++11 is used so we need to add -DCMAKE_CXX_STANDARD=17 to the CMake command.

Expected behavior

I believe the cuBLAS backend should be able to compile using icpx and with recent CMake and CUDA version using a "default" CMake command like:

$ cmake -Bbuild-blas-cuda -GNinja -DTARGET_DOMAINS=blas -DENABLE_MKLCPU_BACKEND=OFF -DENABLE_MKLGPU_BACKEND=OFF -DENABLE_CUBLAS_BACKEND=ON -DBUILD_FUNCTIONAL_TESTS=OFF .
$ ninja -Cbuild-blas-cuda

[Rbiessy]

Tagging @muhammad-tanvir-1211 and @mkrainiuk in case one of you have more information on this issue!

[s-Nick]

Thank you @Rbiessy for highlighting it. Regarding the first observation, right now the cmake cannot find the necessary headers. This is due to the deprecation of clang support in the 2024 oneAPI release and subsequent change of clang++ directory and adding an extra layer. For now, it is possible to work around it adding another hint in cmake/FindcuBLAS.cmake at line 28, add ${SYCL_BINARY_DIR}/../../include/sycl. Updating the compiler from clang++ to icpx would require further investigation and modification in cmake, but the minimum changes required would be to modify the following line: https://github.com/oneapi-src/oneMKL/blob/develop/CMakeLists.txt#L138-L140

[Rbiessy]

Thanks for the quick answer. Do you know if there is any plan to properly fix these issues?