torchmcubes icon indicating copy to clipboard operation
torchmcubes copied to clipboard

Error when building on cuda12.6 nvToolsExt not found

Open UradaSources opened this issue 8 months ago • 1 comments

I used the following command to build: pip install git+https://github.com/tatsy/torchmcubes.git

However, CMake throws an error. Here are parts of the log:

      2025-03-14 14:48:10,054 - scikit_build_core - WARNING - cmake should not be in build-system.requires - scikit-build-core will inject it as needed
      2025-03-14 14:48:10,054 - scikit_build_core - WARNING - ninja should not be in build-system.requires - scikit-build-core will inject it as needed
      2025-03-14 14:48:10,078 - scikit_build_core - INFO - RUN: C:\Users\Administrator\AppData\Local\Temp\pip-build-env-zb0_l3nv\overlay\Lib\site-packages\cmake\data\bin\cmake -E capabilities
      2025-03-14 14:48:10,143 - scikit_build_core - INFO - CMake version: 3.31.6
      *** scikit-build-core 0.11.0 using CMake 3.31.6 (wheel)
      2025-03-14 14:48:10,157 - scikit_build_core - INFO - Build directory: C:\Users\Administrator\AppData\Local\Temp\pip-req-build-2ckbxhqw\build
      *** Configuring CMake...
      2025-03-14 14:48:10,252 - scikit_build_core - WARNING - Can't find a Python library, got libdir=None, ldlibrary=None, multiarch=None, masd=None
      2025-03-14 14:48:10,254 - scikit_build_core - INFO - RUN: C:\Users\Administrator\AppData\Local\Temp\pip-build-env-zb0_l3nv\overlay\Lib\site-packages\cmake\data\bin\cmake -S. -Bbuild -Cbuild\CMakeInit.txt -DCMAKE_INSTALL_PREFIX=C:\Users\ADMINI~1\AppData\Local\Temp\tmplhme7zgq\wheel\platlib
      loading initial cache file build\CMakeInit.txt
      -- Building for: Visual Studio 17 2022
      -- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.19045.
      -- The CXX compiler identification is MSVC 19.41.34123.0
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.41.34120/bin/Hostx64/x64/cl.exe - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Looking for a CUDA compiler
      -- Looking for a CUDA compiler - C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6/bin/nvcc.exe
      -- The CUDA compiler identification is NVIDIA 12.6.20 with host compiler MSVC 19.41.34123.0
      -- Detecting CUDA compiler ABI info
      -- Detecting CUDA compiler ABI info - done
      -- Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6/bin/nvcc.exe - skipped
      -- Detecting CUDA compile features
      -- Detecting CUDA compile features - done
      -- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6/include (found version "12.6.20")
      -- INSTALLING EXTENSIONS WITH CUDA!
      -- CMAKE_CUDA_COMPILER = C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6/bin/nvcc.exe
      -- CMAKE_CUDA_COMPILER_ID = NVIDIA
      -- CMAKE_CUDA_COMPILER_VERSION = 126
      -- Found Python: D:\lch\project\llm\tripo_local\env\Scripts\python.exe (found version "3.11.4") found components: Interpreter Development.Module
      -- Performing Test HAS_MSVC_GL_LTCG
      -- Performing Test HAS_MSVC_GL_LTCG - Success
      -- Found pybind11: C:/Users/Administrator/AppData/Local/Temp/pip-build-env-zb0_l3nv/overlay/Lib/site-packages/pybind11/include (found version "2.13.6")
      -- Found OpenMP_CXX: -openmp (found version "2.0")
      -- Could NOT find OpenMP_CUDA (missing: OpenMP_CUDA_FLAGS OpenMP_CUDA_LIB_NAMES)
      -- Could NOT find OpenMP (missing: OpenMP_CUDA_FOUND) (found version "2.0")
      -- Found CUDA: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6 (found version "12.6")
      -- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6/include (found version "12.6.20")
      -- PyTorch: CUDA detected: 12.6
      -- PyTorch: CUDA nvcc is: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6/bin/nvcc.exe
      -- PyTorch: CUDA toolkit directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.6
      -- PyTorch: Header version is: 12.6
      -- Found Python: D:\lch\project\llm\tripo_local\env\Scripts\python.exe (found version "3.11.4") found components: Interpreter
      CMake Warning at D:/lch/Project/llm/tripo_local/env/Lib/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:140 (message):
        Failed to compute shorthash for libnvrtc.so
      Call Stack (most recent call first):
        D:/lch/Project/llm/tripo_local/env/Lib/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
        D:/lch/Project/llm/tripo_local/env/Lib/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
        CMakeLists.txt:43 (find_package)


      CMake Warning (dev) at C:/Users/Administrator/AppData/Local/Temp/pip-build-env-zb0_l3nv/overlay/Lib/site-packages/cmake/data/share/cmake-3.31/Modules/FindPackageHandleStandardArgs.cmake:441 (message):
        The package name passed to `find_package_handle_standard_args` (nvtx3) does
        not match the name of the calling package (Caffe2).  This can lead to
        problems in calling code that expects `find_package` result variables
        (e.g., `_FOUND`) to follow a certain pattern.
      Call Stack (most recent call first):
        D:/lch/Project/llm/tripo_local/env/Lib/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:178 (find_package_handle_standard_args)
        D:/lch/Project/llm/tripo_local/env/Lib/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
        D:/lch/Project/llm/tripo_local/env/Lib/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
        CMakeLists.txt:43 (find_package)
      This warning is for project developers.  Use -Wno-dev to suppress it.

      -- Could NOT find nvtx3 (missing: nvtx3_dir)
      CMake Warning at D:/lch/Project/llm/tripo_local/env/Lib/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:184 (message):
        Cannot find NVTX3, find old NVTX instead
      Call Stack (most recent call first):
        D:/lch/Project/llm/tripo_local/env/Lib/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
        D:/lch/Project/llm/tripo_local/env/Lib/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
        CMakeLists.txt:43 (find_package)


      -- USE_CUDNN is set to 0. Compiling without cuDNN support
      -- USE_CUSPARSELT is set to 0. Compiling without cuSPARSELt support
      -- USE_CUDSS is set to 0. Compiling without cuDSS support
      -- USE_CUFILE is set to 0. Compiling without cuFile support
      -- Autodetected CUDA architecture(s):  6.1
      -- Added CUDA NVCC flags for: -gencode;arch=compute_61,code=sm_61
      -- Found Torch: D:/lch/Project/llm/tripo_local/env/Lib/site-packages/torch/lib/torch.lib
      -- Configuring done (60.0s)
 (set_property):
        The link interface of target "torch::nvtoolsext" contains:

          CUDA::nvToolsExt

        but the target was not found.  Possible reasons include:

          * There is a typo in the target name.
          * A find_package call is missing for an IMPORTED target.
          * An ALIAS target is missing.

      Call Stack (most recent call first):
        D:/lch/Project/llm/tripo_local/env/Lib/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
        D:/lch/Project/llm/tripo_local/env/Lib/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
        CMakeLists.txt:43 (find_package)


      -- Generating done (0.1s)
      CMake Generate step failed.  Build files cannot be regenerated correctly.

      *** CMake configuration failed
      FAILURE!!
      [end of output]

I am building in a venv and have installed the PyTorch version that matches CUDA, and I have tested:

cuda_available = torch.cuda.is_available()
cuda_device_count = torch.cuda.device_count()

Everything is normal.

CUDA_HOME is correctly set. Here’s my nvcc version:

(env) PS D:\lch\Project\llm\tripo_local> nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Fri_Jun_14_16:44:19_Pacific_Daylight_Time_2024
Cuda compilation tools, release 12.6, V12.6.20
Build cuda_12.6.r12.6/compiler.34431801_0

Additionally, I checked my CUDA installation path and found that the NVTX header files do exist: CUDA\v12.6\include\nvtx3\nvToolsExt.h

I found this: https://discuss.pytorch.org/t/failed-to-find-nvtoolsext/179635/9

It seems that CUDA modified its directory structure after version 12.x. Perhaps the CMakeLists for torchmcubes also needs to be modified?

UradaSources avatar Mar 14 '25 07:03 UradaSources

Strange, it suddenly worked. I don't know what I did. I just copied a nvToolsExt64_1.dll to CUDA\v12.6\lib\x64, but I don't understand why this makes find_package successful

UradaSources avatar Mar 14 '25 07:03 UradaSources