MinkowskiEngine icon indicating copy to clipboard operation
MinkowskiEngine copied to clipboard

I ran into almost every issue in installing and I wrote a troubleshooting guide (both CUDA 11 & 12 supported)

Open CiSong10 opened this issue 7 months ago • 9 comments

I've installed MinkowskiEngine more times than I can count this week because I hate my life, running into just about every issue imaginable. I put together some notes for myself, but I figured they might be helpful to others as well. These steps should work for building with both CUDA 11 and CUDA 12.

MinkowskiEngine is a powerful library, and I’m genuinely grateful to the developers for their work. That said, last commit was 2 years ago, which makes me wonder if the developers are still maintaining it. I really hope it gets some much-needed upgrades.

Here is the link:

https://www.youtube.com/watch?v=Q4SWxWIOVBM

Sorry, wrong link:

https://github.com/CiSong10/MinkowskiEngine/blob/cuda12-installation/installation_note.md

CiSong10 avatar May 21 '25 18:05 CiSong10

I write a docker file with the necessary fix you could merge into your repo if you like @CiSong10

ARG PYTORCH="2.7.0"
ARG CUDA="12.8"
ARG CUDNN="9"

FROM pytorch/pytorch:${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel
ENV TORCH_NVCC_FLAGS="-Xfatbin -compress-all"

ARG CUDA_SM_LIST="6.0 6.1 6.2 7.0 7.2 7.5 8.0 8.6 8.9 9.0 12.0"
# 2) Export it for CMake (>=3.18) and PyTorch builds:
ENV CMAKE_CUDA_ARCHITECTURES=${CUDA_SM_LIST} \
    TORCH_CUDA_ARCH_LIST=${CUDA_SM_LIST}


# Install dependencies
RUN apt-get update
RUN apt-get install -y git ninja-build cmake build-essential libopenblas-dev \
    xterm xauth openssh-server tmux wget mate-desktop-environment-core

WORKDIR /workspace
RUN git clone https://github.com/CiSong10/MinkowskiEngine.git && \
    cd MinkowskiEngine && \
    git checkout cuda12-installation 

WORKDIR ./MinkowskiEngine  
# Apply the fix for error: NVTX3 header collisions (https://github.com/NVIDIA/MinkowskiEngine/issues/614#issuecomment-2886009673)
RUN sed -i '/sources=.*BIND_FILES.*/a\        define_macros=[('"'"'NVTX_DISABLE'"'"', None)],' setup.py 
# Apply the fix for error: Compilation: compiler finds both std::to_address and cuda::std::to_address (https://github.com/NVIDIA/MinkowskiEngine/issues/596)
RUN sed -i 's/\bauto __raw = __to_address(__r.get());/auto __raw = std::__to_address(__r.get());/' /usr/include/c++/11/bits/shared_ptr_base.h
RUN python setup.py install --force_cuda --blas=openblas

QuteSaltyFish avatar Jul 28 '25 17:07 QuteSaltyFish

Awesome solution. but sth wrong with my CUDA 12.5 so I just get back to CUDA 12.2.

Lans1ot avatar Aug 07 '25 12:08 Lans1ot

@QuteSaltyFish Thank you. I have pushed this change into my branch. I also pushed the fix for NVTX3 header collisions in my branch as well, so there's no need for this step now:

# Apply the fix for error: NVTX3 header collisions (https://github.com/NVIDIA/MinkowskiEngine/issues/614#issuecomment-2886009673)
RUN sed -i '/sources=.*BIND_FILES.*/a\        define_macros=[('"'"'NVTX_DISABLE'"'"', None)],' setup.py 

-- shorter Dockerfile ,yay

CiSong10 avatar Aug 07 '25 19:08 CiSong10

I write a docker file with the necessary fix you could merge into your repo if you like @CiSong10

ARG PYTORCH="2.7.0"
ARG CUDA="12.8"
ARG CUDNN="9"

FROM pytorch/pytorch:${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel
ENV TORCH_NVCC_FLAGS="-Xfatbin -compress-all"

ARG CUDA_SM_LIST="6.0 6.1 6.2 7.0 7.2 7.5 8.0 8.6 8.9 9.0 12.0"
# 2) Export it for CMake (>=3.18) and PyTorch builds:
ENV CMAKE_CUDA_ARCHITECTURES=${CUDA_SM_LIST} \
    TORCH_CUDA_ARCH_LIST=${CUDA_SM_LIST}


# Install dependencies
RUN apt-get update
RUN apt-get install -y git ninja-build cmake build-essential libopenblas-dev \
    xterm xauth openssh-server tmux wget mate-desktop-environment-core

WORKDIR /workspace
RUN git clone https://github.com/CiSong10/MinkowskiEngine.git && \
    cd MinkowskiEngine && \
    git checkout cuda12-installation 

WORKDIR ./MinkowskiEngine  
# Apply the fix for error: NVTX3 header collisions (https://github.com/NVIDIA/MinkowskiEngine/issues/614#issuecomment-2886009673)
RUN sed -i '/sources=.*BIND_FILES.*/a\        define_macros=[('"'"'NVTX_DISABLE'"'"', None)],' setup.py 
# Apply the fix for error: Compilation: compiler finds both std::to_address and cuda::std::to_address (https://github.com/NVIDIA/MinkowskiEngine/issues/596)
RUN sed -i 's/\bauto __raw = __to_address(__r.get());/auto __raw = std::__to_address(__r.get());/' /usr/include/c++/11/bits/shared_ptr_base.h
RUN python setup.py install --force_cuda --blas=openblas

Hello ! thank u for your work , however, due to specific issues, i am wondering if you could provide a illustration of install ME on host machine? I followd you instruction wihout mamba but used conda and pip instead. Since i am using a 5090, i can only use the torch 2.7.0 or higher versions.

WiGig11 avatar Sep 16 '25 12:09 WiGig11

I write a docker file with the necessary fix you could merge into your repo if you like @CiSong10

ARG PYTORCH="2.7.0"
ARG CUDA="12.8"
ARG CUDNN="9"

FROM pytorch/pytorch:${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel
ENV TORCH_NVCC_FLAGS="-Xfatbin -compress-all"

ARG CUDA_SM_LIST="6.0 6.1 6.2 7.0 7.2 7.5 8.0 8.6 8.9 9.0 12.0"
# 2) Export it for CMake (>=3.18) and PyTorch builds:
ENV CMAKE_CUDA_ARCHITECTURES=${CUDA_SM_LIST} \
    TORCH_CUDA_ARCH_LIST=${CUDA_SM_LIST}


# Install dependencies
RUN apt-get update
RUN apt-get install -y git ninja-build cmake build-essential libopenblas-dev \
    xterm xauth openssh-server tmux wget mate-desktop-environment-core

WORKDIR /workspace
RUN git clone https://github.com/CiSong10/MinkowskiEngine.git && \
    cd MinkowskiEngine && \
    git checkout cuda12-installation 

WORKDIR ./MinkowskiEngine  
# Apply the fix for error: NVTX3 header collisions (https://github.com/NVIDIA/MinkowskiEngine/issues/614#issuecomment-2886009673)
RUN sed -i '/sources=.*BIND_FILES.*/a\        define_macros=[('"'"'NVTX_DISABLE'"'"', None)],' setup.py 
# Apply the fix for error: Compilation: compiler finds both std::to_address and cuda::std::to_address (https://github.com/NVIDIA/MinkowskiEngine/issues/596)
RUN sed -i 's/\bauto __raw = __to_address(__r.get());/auto __raw = std::__to_address(__r.get());/' /usr/include/c++/11/bits/shared_ptr_base.h
RUN python setup.py install --force_cuda --blas=openblas

Hello ! thank u for your work , however, due to specific issues, i am wondering if you could provide a illustration of install ME on host machine? I followd you instruction wihout mamba but used conda and pip instead. Since i am using a 5090, i can only use the torch 2.7.0 or higher versions.

You can do the same setup using the RUN command. If you're using conda, you can first conda install nvidia/label/cuda-12.8.1::cuda and then install the pytorch 2.7. Then you could run the command I write in the dockerfile to build ME.

QuteSaltyFish avatar Sep 16 '25 18:09 QuteSaltyFish

I write a docker file with the necessary fix you could merge into your repo if you like @CiSong10

ARG PYTORCH="2.7.0"
ARG CUDA="12.8"
ARG CUDNN="9"

FROM pytorch/pytorch:${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel
ENV TORCH_NVCC_FLAGS="-Xfatbin -compress-all"

ARG CUDA_SM_LIST="6.0 6.1 6.2 7.0 7.2 7.5 8.0 8.6 8.9 9.0 12.0"
# 2) Export it for CMake (>=3.18) and PyTorch builds:
ENV CMAKE_CUDA_ARCHITECTURES=${CUDA_SM_LIST} \
    TORCH_CUDA_ARCH_LIST=${CUDA_SM_LIST}


# Install dependencies
RUN apt-get update
RUN apt-get install -y git ninja-build cmake build-essential libopenblas-dev \
    xterm xauth openssh-server tmux wget mate-desktop-environment-core

WORKDIR /workspace
RUN git clone https://github.com/CiSong10/MinkowskiEngine.git && \
    cd MinkowskiEngine && \
    git checkout cuda12-installation 

WORKDIR ./MinkowskiEngine  
# Apply the fix for error: NVTX3 header collisions (https://github.com/NVIDIA/MinkowskiEngine/issues/614#issuecomment-2886009673)
RUN sed -i '/sources=.*BIND_FILES.*/a\        define_macros=[('"'"'NVTX_DISABLE'"'"', None)],' setup.py 
# Apply the fix for error: Compilation: compiler finds both std::to_address and cuda::std::to_address (https://github.com/NVIDIA/MinkowskiEngine/issues/596)
RUN sed -i 's/\bauto __raw = __to_address(__r.get());/auto __raw = std::__to_address(__r.get());/' /usr/include/c++/11/bits/shared_ptr_base.h
RUN python setup.py install --force_cuda --blas=openblas

Hello ! thank u for your work , however, due to specific issues, i am wondering if you could provide a illustration of install ME on host machine? I followd you instruction wihout mamba but used conda and pip instead. Since i am using a 5090, i can only use the torch 2.7.0 or higher versions.

You can do the same setup using the RUN command. If you're using conda, you can first conda install nvidia/label/cuda-12.8.1::cuda and then install the pytorch 2.7. Then you could run the command I write in the dockerfile to build ME.

Hello! so sorry to bother you once more. However, as requested, i write a script based on the suggestions of GPT-5 and you. Here is my script and the response i get:

 ```bash #!/usr/bin/env bash ######################################## # Customizable parameters ######################################## ENV_NAME="mink" PYTHON_VERSION="3.10" PYTORCH_VERSION="2.7.0" CUDA_CONDA_VERSION="12.4.0" # CUDA toolkit available in conda (does not rely on system CUDA) WORKDIR="${HOME}/workspace" # CUDA architecture list matching Dockerfile CUDA_SM_LIST="6.0 6.1 6.2 7.0 7.2 7.5 8.0 8.6 8.9 9.0 12.0" ######################################## # 0) Prepare conda ######################################## if ! command -v conda >/dev/null 2>&1; then echo "[ERROR] Conda not detected, please install Miniconda/Anaconda first." exit 1 fi eval "$(conda shell.bash hook)" conda deactivate || true if ! conda env list | awk '{print $1}' | grep -qx "${ENV_NAME}"; then conda create -y -n "${ENV_NAME}" "python=${PYTHON_VERSION}" fi conda activate "${ENV_NAME}" ######################################## # 1) Install build tools and dependencies only in this environment (conda-forge) # No system package installations (no apt) ######################################## conda install -y -c conda-forge \ cmake ninja pkg-config \ openblas openblas-devel \ gcc_linux-64=12.* gxx_linux-64=12.* \ wget unzip ######################################## # 2) Install CUDA toolkit (including nvcc) only in this environment # Completely rely on the CUDA toolkit from conda ######################################## conda install -y -c "nvidia/label/cuda-${CUDA_CONDA_VERSION}" cuda-toolkit ######################################## # 3) Create activation script for this environment (only affects this environment) # Does not modify ~/.bashrc or affect global environment ######################################## mkdir -p "$CONDA_PREFIX/etc/conda/activate.d" cat > "$CONDA_PREFIX/etc/conda/activate.d/99_mink_env.sh" /dev/null || true wget -q https://github.com/CiSong10/MinkowskiEngine/archive/refs/heads/cuda12-installation.zip unzip -q cuda12-installation.zip cd MinkowskiEngine-cuda12-installation ######################################## # 6) Apply patches (only affects this environment/project, no system changes) ######################################## # 6.1 NVTX3 conflict: Insert define_macros in setup.py (same as Dockerfile) sed -i "/sources=.*BIND_FILES.*/a\ define_macros=[('NVTX_DISABLE', None)]," setup.py # 6.2 to_address conflict: Only patch the header files in the conda sysroot (no system modification) HEADER_CONDA=$(python - 

response: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/data/test/MinkowskiEngine-cuda12-installation/MinkowskiEngine/__init__.py", line 53, in <module> from MinkowskiEngineBackend._C import ( ModuleNotFoundError: No module named 'MinkowskiEngineBackend'

WiGig11 avatar Sep 17 '25 02:09 WiGig11

I write a docker file with the necessary fix you could merge into your repo if you like @CiSong10

ARG PYTORCH="2.7.0"
ARG CUDA="12.8"
ARG CUDNN="9"

FROM pytorch/pytorch:${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel
ENV TORCH_NVCC_FLAGS="-Xfatbin -compress-all"

ARG CUDA_SM_LIST="6.0 6.1 6.2 7.0 7.2 7.5 8.0 8.6 8.9 9.0 12.0"
# 2) Export it for CMake (>=3.18) and PyTorch builds:
ENV CMAKE_CUDA_ARCHITECTURES=${CUDA_SM_LIST} \
    TORCH_CUDA_ARCH_LIST=${CUDA_SM_LIST}


# Install dependencies
RUN apt-get update
RUN apt-get install -y git ninja-build cmake build-essential libopenblas-dev \
    xterm xauth openssh-server tmux wget mate-desktop-environment-core

WORKDIR /workspace
RUN git clone https://github.com/CiSong10/MinkowskiEngine.git && \
    cd MinkowskiEngine && \
    git checkout cuda12-installation 

WORKDIR ./MinkowskiEngine  
# Apply the fix for error: NVTX3 header collisions (https://github.com/NVIDIA/MinkowskiEngine/issues/614#issuecomment-2886009673)
RUN sed -i '/sources=.*BIND_FILES.*/a\        define_macros=[('"'"'NVTX_DISABLE'"'"', None)],' setup.py 
# Apply the fix for error: Compilation: compiler finds both std::to_address and cuda::std::to_address (https://github.com/NVIDIA/MinkowskiEngine/issues/596)
RUN sed -i 's/\bauto __raw = __to_address(__r.get());/auto __raw = std::__to_address(__r.get());/' /usr/include/c++/11/bits/shared_ptr_base.h
RUN python setup.py install --force_cuda --blas=openblas

in order for my MinkowskiEngine to work on

nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2025 NVIDIA Corporation Built on Fri_Feb_21_20:23:50_PST_2025 Cuda compilation tools, release 12.8, V12.8.93 Build cuda_12.8.r12.8/compiler.35583870_0 CuDNN 9 Python 3.10.12 torch == 2.7.1

i had to only apply the fix for the #fix for error: NVTX3 header collisions, which are the sed lines for replacing text, worked like a charm ! much obliged! 💯

eyegotthis avatar Nov 05 '25 16:11 eyegotthis

Thank you for the docker. However, I got this error when run it

python setup.py install --force_cuda --blas=openblas File "/working/MinkowskiEngine/setup.py", line 315
define_macros=[('NVTX_DISABLE', None)],
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ SyntaxError: keyword argument repeated: define_macros

letatanu avatar Nov 09 '25 18:11 letatanu

Thank you for the docker. However, I got this error when run it

python setup.py install --force_cuda --blas=openblas File "/working/MinkowskiEngine/setup.py", line 315 define_macros=[('NVTX_DISABLE', None)], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ SyntaxError: keyword argument repeated: define_macros

Hey @letatanu The solution is simple, the file seems to have the same line repeated use the following after

RUN sed -i '/sources=.BIND_FILES./a\ define_macros=[('"'"'NVTX_DISABLE'"'"', None)],' setup.py

RUN sed -i '315d' setup.py

eyegotthis avatar Nov 11 '25 09:11 eyegotthis