llama-cpp-python icon indicating copy to clipboard operation
llama-cpp-python copied to clipboard

See `No CUDA toolset found` log after `-- CUDA Toolkit found.` log

Open TomaszDlubis opened this issue 10 months ago • 1 comments

I have installed CUDA in version 12.5, which is detected by CMake and is visible in the terminal. The CUDA_PATH is also correctly set. During trying to install, I see those logs:

pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu125
...
     -- Performing Test HAS_FMA_1 - Success
      -- Performing Test HAS_AVX512_1
      -- Performing Test HAS_AVX512_1 - Failed
      -- Performing Test HAS_AVX512_2
      -- Performing Test HAS_AVX512_2 - Failed
      -- Adding CPU backend variant ggml-cpu: /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C
      -- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.5/include (found version "12.5.40")
      -- CUDA Toolkit found
      -- Using CUDA architectures: native
      CMake Error at C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.29/Modules/CMakeDetermineCompilerId.cmake:563 (message):
        No CUDA toolset found.
      Call Stack (most recent call first):
        C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.29/Modules/CMakeDetermineCompilerId.cmake:8 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
        C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.29/Modules/CMakeDetermineCompilerId.cmake:53 (__determine_compiler_id_test)
        C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.29/Modules/CMakeDetermineCUDACompiler.cmake:131 (CMAKE_DETERMINE_COMPILER_ID)
        vendor/llama.cpp/ggml/src/ggml-cuda/CMakeLists.txt:25 (enable_language)


      -- Configuring incomplete, errors occurred!

      *** CMake configuration failed
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Failed to build installable wheels for some pyproject.toml based projects (llama-cpp-python)

My environment: Windows 11, Cuda12.5, Python 3.12.8, ninja 1.11.1.3, cmake 3.31.4

TomaszDlubis avatar Feb 03 '25 11:02 TomaszDlubis