CMake Picks Up Wrong `nvcc` Path During Build

[bhy]

Prerequisites

• [x] I am running the latest code.
• [x] I carefully followed the README.md.
• [x] I have searched for similar issues and found none.
• [x] I reviewed the Discussions, and believe this is a new bug.

Expected Behavior

During the build process, I expected llama.cpp to correctly pick up the path of the nvcc CUDA compiler that is in my $PATH, (ie. which nvcc).

Current Behavior

The build process with CMake picks up a wrong path for the nvcc CUDA compiler. The path is /usr/bin/nvcc, while the correct path in my environment is .../conda/env/bin/nvcc.

Environment and Context

• Hardware: Nvidia P100 GPU
• Operating System: Ubuntu 20.04 with conda environment setup by text-generation-webui

Steps to Reproduce

1. Set up the conda environment with the installation script from text-generation-webui project.
2. checkout latest llama.cpp (commit 74a6d92).
3. Run cmake .. -DLLAMA_CUBLAS=ON.
4. Observe the output from CMake, which incorrectly detects the nvcc path as /usr/bin/nvcc.

Failure Logs

During the build configuration process with CMake, the following output is observed:

-- Check for working CUDA compiler: /usr/bin/nvcc - skipped

However, the correct path for nvcc is .../conda/env/bin/nvcc.

This will then cause many downstream build errors such as

ptxas fatal : Value 'sm_30' is not defined for option 'gpu-name'                                                                                                                                                                                          

or

nvcc fatal   : 'f16c': expected a number

Workaround

cmake .. -DLLAMA_CUBLAS=ON -DCMAKE_CUDA_COMPILER=$(which nvcc)

[slaren]

I don't think that this is a bug. The search behavior of cmake for the CUDA toolkit is documented here. For non-standard paths, your workaround seems good enough. Setting the CUDA_PATH environment variable may also work.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.