pyg-lib icon indicating copy to clipboard operation
pyg-lib copied to clipboard

Published 20 hours ago • verified publisher icon pyg-team

Success install but import error

Open ShiyinTan opened this issue 1 year ago • 16 comments

🐛 Describe the bug

import torch 
import pyg_lib

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/X/anaconda3/envs/py3.10/lib/python3.10/site-packages/pyg_lib/__init__.py", line 39, in <module>
    load_library('libpyg')
  File "/home/X/anaconda3/envs/py3.10/lib/python3.10/site-packages/pyg_lib/__init__.py", line 36, in load_library
    torch.ops.load_library(spec.origin)
  File "/home/X/anaconda3/envs/py3.10/lib/python3.10/site-packages/torch/_ops.py", line 852, in load_library
    ctypes.CDLL(path)
  File "/home/X/anaconda3/envs/py3.10/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /home/X/anaconda3/envs/py3.10/lib/python3.10/site-packages/libpyg.so)

Environment

  • pyg-lib version: 0.3.0
  • PyTorch version: 2.1.0
  • OS: Ubuntu 18.04
  • Python version: 3.10
  • CUDA/cuDNN version:CUDA 11.80, cuDNN
  • How you installed PyTorch and pyg-lib (conda, pip, source): Pytorch: conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia pyg-lib: pip install pyg-lib -f https://data.pyg.org/whl/torch-2.1.0+cu118.html
  • Any other relevant information: The error information are:

My GLIBC version is

(Ubuntu GLIBC 2.27-3ubuntu1) 2.27

I'm quite sure that I can't upgrade my GLIBC version, how can I fix this issue? Is this a bug or a Compatibility issues? I'm not sure.

Thank you very much for help.

ShiyinTan avatar Oct 17 '23 10:10 ShiyinTan

Thanks for reporting. Looks like our wheels somehow require GLIBC=2.29. I need to investigate. In the meantime, you should have no problem to install pyg-lib from source directly:

pip install git+https://github.com/pyg-team/pyg-lib.git

rusty1s avatar Oct 20 '23 15:10 rusty1s

I have the same problem.

jasong-ovo avatar Oct 23 '23 12:10 jasong-ovo

Facing the same issue, details below:

Environment

  • pyg-lib version: 0.3.0+pt21cu118
  • PyTorch version: 2.1.0+cu118
  • OS: Red Hat Enterprise Linux Server 7.9 Maipo (Using Developer Toolset-9.0)
  • Python version: 3.11.0
  • Installation method:
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install torch_geometric
pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-2.1.0+cu118.html
  • glibc : 2.17

Complete Error:

/home/../miniconda3/envs/pyg/lib/python3.11/site-packages/torch_geometric/typing.py:47: UserWarning: An issue occurred while importing 'pyg-lib'. Disabling its usage. Stacktrace: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by /home/../miniconda3/envs/pyg1/lib/python3.11/site-packages/libpyg.so)
  warnings.warn(f"An issue occurred while importing 'pyg-lib'. "

inkandvijay avatar Oct 31 '23 21:10 inkandvijay

Hi,

I tried solving this with pip install git+https://github.com/pyg-team/pyg-lib.git but got the following error:

pip install git+https://github.com/pyg-team/pyg-lib.git
Collecting git+https://github.com/pyg-team/pyg-lib.git
  Cloning https://github.com/pyg-team/pyg-lib.git to /w0/tmp/am232141/login18-g-1_218949/pip-req-build-1omdvorq
  Running command git clone --filter=blob:none --quiet https://github.com/pyg-team/pyg-lib.git /w0/tmp/am232141/login18-g-1_218949/pip-req-build-1omdvorq
  Resolved https://github.com/pyg-team/pyg-lib.git to commit 44760ec4e3591a9238393457725fa07ba7bd4491
  Running command git submodule update --init --recursive -q
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: pyg-lib
  Building wheel for pyg-lib (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [91 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-39
      creating build/lib.linux-x86_64-cpython-39/pyg_lib
      copying pyg_lib/__init__.py -> build/lib.linux-x86_64-cpython-39/pyg_lib
      copying pyg_lib/_triton.py -> build/lib.linux-x86_64-cpython-39/pyg_lib
      copying pyg_lib/home.py -> build/lib.linux-x86_64-cpython-39/pyg_lib
      copying pyg_lib/testing.py -> build/lib.linux-x86_64-cpython-39/pyg_lib
      creating build/lib.linux-x86_64-cpython-39/pyg_lib/ops
      copying pyg_lib/ops/__init__.py -> build/lib.linux-x86_64-cpython-39/pyg_lib/ops
      copying pyg_lib/ops/scatter_reduce.py -> build/lib.linux-x86_64-cpython-39/pyg_lib/ops
      creating build/lib.linux-x86_64-cpython-39/pyg_lib/partition
      copying pyg_lib/partition/__init__.py -> build/lib.linux-x86_64-cpython-39/pyg_lib/partition
      creating build/lib.linux-x86_64-cpython-39/pyg_lib/sampler
      copying pyg_lib/sampler/__init__.py -> build/lib.linux-x86_64-cpython-39/pyg_lib/sampler
      running build_ext
      /w0/tmp/am232141/login18-g-1_218949/pip-req-build-1omdvorq/setup.py:77: UserWarning: Building times of 'pyg-lib' can be heavily improved by installing 'ninja': `pip install ninja`
        warnings.warn("Building times of 'pyg-lib' can be heavily improved"
      -- The C compiler identification is Intel 2021.6.0.20220226
      -- The CXX compiler identification is Intel 2021.6.0.20220226
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /cvmfs/software.hpc.rwth.de/Linux/RH8/x86_64/intel/skylake_avx512/software/intel-compilers/2022.1.0/compiler/2022.1.0/linux/bin/intel64/icc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /cvmfs/software.hpc.rwth.de/Linux/RH8/x86_64/intel/skylake_avx512/software/intel-compilers/2022.1.0/compiler/2022.1.0/linux/bin/intel64/icpc - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Found Python3: /rwthfs/rz/cluster/home/am232141/miniconda3/envs/mlpfenv_test/include/python3.9 (found version "3.9.18") found components: Development Development.Module Development.Embed
      CMake Error at /usr/share/cmake/Modules/CMakeDetermineCUDACompiler.cmake:174 (message):
        Failed to find nvcc.

        Compiler requires the CUDA toolkit.  Please set the CUDAToolkit_ROOT
        variable.
      Call Stack (most recent call first):
        CMakeLists.txt:35 (enable_language)


      -- Configuring incomplete, errors occurred!
      See also "/w0/tmp/am232141/login18-g-1_218949/pip-req-build-1omdvorq/build/temp.linux-x86_64-cpython-39/CMakeFiles/CMakeOutput.log".
      See also "/w0/tmp/am232141/login18-g-1_218949/pip-req-build-1omdvorq/build/temp.linux-x86_64-cpython-39/CMakeFiles/CMakeError.log".
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/w0/tmp/am232141/login18-g-1_218949/pip-req-build-1omdvorq/setup.py", line 130, in <module>
          setup(
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/__init__.py", line 107, in setup
          return distutils.core.setup(**attrs)
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
          super().run_command(command)
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 364, in run
          self.run_command("build")
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
          super().run_command(command)
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
          super().run_command(command)
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 84, in run
          _build_ext.run(self)
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "/w0/tmp/am232141/login18-g-1_218949/pip-req-build-1omdvorq/setup.py", line 82, in build_extension
          subprocess.check_call(['cmake', ext.sourcedir] + cmake_args,
        File "/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/subprocess.py", line 373, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['cmake', '/w0/tmp/am232141/login18-g-1_218949/pip-req-build-1omdvorq', '-DBUILD_TEST=OFF', '-DBUILD_BENCHMARK=OFF', '-DUSE_PYTHON=ON', '-DWITH_CUDA=ON', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/w0/tmp/am232141/login18-g-1_218949/pip-req-build-1omdvorq/build/lib.linux-x86_64-cpython-39', '-DCMAKE_BUILD_TYPE=RELEASE', '-DCMAKE_PREFIX_PATH=/home/am232141/miniconda3/envs/mlpfenv_test/lib/python3.9/site-packages/torch/share/cmake']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyg-lib
  Running setup.py clean for pyg-lib
Failed to build pyg-lib
ERROR: Could not build wheels for pyg-lib, which is required to install pyproject.toml-based projects

viktor-ktorvi avatar Nov 03 '23 22:11 viktor-ktorvi

The issue is that you don't have a CUDA toolkit installed:

Failed to find nvcc.

        Compiler requires the CUDA toolkit.  Please set the CUDAToolkit_ROOT
        variable.

I will try to provide updated binaries today or tomorrow.

rusty1s avatar Nov 06 '23 10:11 rusty1s

Just ran into this as well

benf549 avatar Nov 09 '23 04:11 benf549

I have the same issue, is there any solution? Also, looks like the latest pytorch version is 2.1.1, is it compatible with pyg_lib? thanks

yuanchaohu avatar Nov 20 '23 01:11 yuanchaohu

Also, looks like the latest pytorch version is 2.1.1, is it compatible with pyg_lib? thanks

Yes.

Regarding the other issue: Is it resolved by https://github.com/pyg-team/pyg-lib/issues/266#issuecomment-1794538047?

rusty1s avatar Nov 20 '23 08:11 rusty1s

Regarding the other issue: Is it resolved by #266 (comment)?

not resolved yet, it is not working for mac system, but looks fine for linux. is there anything else to pay attention to for mac system? thanks

yuanchaohu avatar Nov 20 '23 16:11 yuanchaohu

For Mac, the CUDA code path should never get called, so it is very surprising to me it searches for nvcc. What happens if you try to install via

FORCE_CUDA=0 pip install ...

rusty1s avatar Nov 21 '23 08:11 rusty1s

I am having the same issue. I am on a Linux system with GLIBC 2.28. I don't have this issue if I install only torch_scatter and torch_sparse alongside torch_geometric, but I was prompted to install pyg_lib by this warning:

UserWarning: Using 'NeighborSampler' without a 'pyg-lib' installation is deprecated and will be removed soon. Please install 'pyg-lib' for accelerated neighborhood sampling

I am able to reproduce this error by just running import pyg_lib:

OSError: /lib64/libm.so.6: version `GLIBC_2.29' not found

~My environment is created with conda install pytorch=2.2.1 pytorch-cuda=11.8 -c pytorch -c nvidia and pip install pyg_lib torch_scatter torch_sparse -f https://data.pyg.org/whl/torch-2.2.0+cpu.html~

EDIT: My environment is created with conda install pytorch=2.2.1 pytorch-cuda=11.8 -c pytorch -c nvidia and pip install pyg_lib torch_scatter torch_sparse -f https://data.pyg.org/whl/torch-2.2.0+cu118.html

FrenkT avatar Mar 01 '24 12:03 FrenkT

You should run

conda install pytorch=2.2.1 pytorch-cuda=11.8 -c pytorch -c nvidia
pip install pyg_lib torch_scatter torch_sparse -f https://data.pyg.org/whl/torch-2.2.0+cu118.html

rusty1s avatar Mar 01 '24 15:03 rusty1s

You should run

conda install pytorch=2.2.1 pytorch-cuda=11.8 -c pytorch -c nvidia
pip install pyg_lib torch_scatter torch_sparse -f https://data.pyg.org/whl/torch-2.2.0+cu118.html

Yes, I actually used cu118. It's just a typo in the post above (edited now, sorry for the confusion).

FrenkT avatar Mar 01 '24 15:03 FrenkT

Got it, sorry :)

As a workaround, you can do manual installation of pyg-lib via

pip install ninja wheel cmake
pip install git+https://github.com/pyg-team/pyg-lib.git

It looks like pre-built pyg-lib now requires GLIBC_2.29 and I need to understand why :)

rusty1s avatar Mar 01 '24 16:03 rusty1s

I'm trying to get pyg-lib installed on a HPC, but getting the below error when I do:

pip install ninja wheel cmake
pip install git+https://github.com/pyg-team/pyg-lib.git

ERROR:

Building wheels for collected packages: pyg_lib
  Building wheel for pyg_lib (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [102 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-38
      creating build/lib.linux-x86_64-cpython-38/pyg_lib
      copying pyg_lib/__init__.py -> build/lib.linux-x86_64-cpython-38/pyg_lib
      copying pyg_lib/_triton.py -> build/lib.linux-x86_64-cpython-38/pyg_lib
      copying pyg_lib/home.py -> build/lib.linux-x86_64-cpython-38/pyg_lib
      copying pyg_lib/testing.py -> build/lib.linux-x86_64-cpython-38/pyg_lib
      creating build/lib.linux-x86_64-cpython-38/pyg_lib/ops
      copying pyg_lib/ops/__init__.py -> build/lib.linux-x86_64-cpython-38/pyg_lib/ops
      copying pyg_lib/ops/scatter_reduce.py -> build/lib.linux-x86_64-cpython-38/pyg_lib/ops
      creating build/lib.linux-x86_64-cpython-38/pyg_lib/partition
      copying pyg_lib/partition/__init__.py -> build/lib.linux-x86_64-cpython-38/pyg_lib/partition
      creating build/lib.linux-x86_64-cpython-38/pyg_lib/sampler
      copying pyg_lib/sampler/__init__.py -> build/lib.linux-x86_64-cpython-38/pyg_lib/sampler
      running build_ext
      -- The C compiler identification is GNU 4.8.5
      -- The CXX compiler identification is GNU 4.8.5
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Found Python3: /share/apps/centos7/anaconda3/2023.03/include/python3.10 (found version "3.10.10") found components: Development Development.Module Development.Embed
      -- Looking for execinfo.h
      -- Looking for execinfo.h - found
      -- Looking for getline
      -- Looking for getline - found
      CUDA_TOOLKIT_ROOT_DIR not found or specified
      -- Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY)
      CMake Warning at /data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:31 (message):
        Caffe2: CUDA cannot be found.  Depending on whether you are building Caffe2
        or a Caffe2 dependent library, the next warning / error will give you more
        info.
      Call Stack (most recent call first):
        /data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:87 (include)
        /data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
        CMakeLists.txt:79 (find_package)
      
      
      CMake Error at /data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:91 (message):
        Your installed Caffe2 version uses CUDA but I cannot find the CUDA
        libraries.  Please set the proper CUDA prefixes and / or install CUDA.
      Call Stack (most recent call first):
        /data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
        CMakeLists.txt:79 (find_package)
      
      
      -- Configuring incomplete, errors occurred!
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-req-build-00p8xg34/setup.py", line 129, in <module>
          setup(
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/__init__.py", line 104, in setup
          return distutils.core.setup(**attrs)
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/dist.py", line 967, in run_command
          super().run_command(command)
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 368, in run
          self.run_command("build")
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/dist.py", line 967, in run_command
          super().run_command(command)
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/dist.py", line 967, in run_command
          super().run_command(command)
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 91, in run
          _build_ext.run(self)
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-req-build-00p8xg34/setup.py", line 82, in build_extension
          subprocess.check_call(['cmake', ext.sourcedir] + cmake_args,
        File "/data/home/bty174/.conda/envs/slp/lib/python3.8/subprocess.py", line 364, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['cmake', '/tmp/pip-req-build-00p8xg34', '-DBUILD_TEST=OFF', '-DBUILD_BENCHMARK=OFF', '-DUSE_PYTHON=ON', '-DWITH_CUDA=OFF', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/tmp/pip-req-build-00p8xg34/build/lib.linux-x86_64-cpython-38', '-DCMAKE_BUILD_TYPE=RELEASE', '-DCMAKE_PREFIX_PATH=/data/home/bty174/.conda/envs/slp/lib/python3.8/site-packages/torch/share/cmake', '-GNinja']' returned non-zero exit status 1.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyg_lib
  Running setup.py clean for pyg_lib
Failed to build pyg_lib
ERROR: Could not build wheels for pyg_lib, which is required to install pyproject.toml-based projects

Any guidance or tips here?

aaronwtr avatar Mar 18 '24 23:03 aaronwtr

Do you want to build with CUDA or without? it looks for a CUDA implementation but cannot find it. If you want to install without CUDA, try

FORCE_CUDA=0 pip install git+https://github.com/pyg-team/pyg-lib.git

rusty1s avatar Mar 20 '24 12:03 rusty1s

@rusty1s Hi thank you for concerning this problems, and i met the same problems.

I used Linux HPC, and install the packages with pip, and i can reproduce this problem OSError: /lib64/libm.so.6: version GLIBC_2.27' not found (required by /public/home/USER/.conda/envs/jptr/lib/python3.8/site-packages/libpyg.so)by sololyimport pyg_lib` in jupyter and command line;

And i met the same problem with @aaronwtr

Building wheels for collected packages: pyg_lib
  Building wheel for pyg_lib (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [102 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-38
      creating build/lib.linux-x86_64-cpython-38/pyg_lib
      copying pyg_lib/home.py -> build/lib.linux-x86_64-cpython-38/pyg_lib
      copying pyg_lib/_triton.py -> build/lib.linux-x86_64-cpython-38/pyg_lib
      copying pyg_lib/__init__.py -> build/lib.linux-x86_64-cpython-38/pyg_lib
      copying pyg_lib/testing.py -> build/lib.linux-x86_64-cpython-38/pyg_lib
      creating build/lib.linux-x86_64-cpython-38/pyg_lib/partition
      copying pyg_lib/partition/__init__.py -> build/lib.linux-x86_64-cpython-38/pyg_lib/partition
      creating build/lib.linux-x86_64-cpython-38/pyg_lib/sampler
      copying pyg_lib/sampler/__init__.py -> build/lib.linux-x86_64-cpython-38/pyg_lib/sampler
      creating build/lib.linux-x86_64-cpython-38/pyg_lib/ops
      copying pyg_lib/ops/scatter_reduce.py -> build/lib.linux-x86_64-cpython-38/pyg_lib/ops
      copying pyg_lib/ops/__init__.py -> build/lib.linux-x86_64-cpython-38/pyg_lib/ops
      running build_ext
      -- The C compiler identification is GNU 4.8.5
      -- The CXX compiler identification is GNU 4.8.5
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Found Python3: /public/home/qiujiangdong/.conda/envs/jptr/include/python3.8 (found version "3.8.19") found components: Development Development.Module Development.Embed
      -- Looking for execinfo.h
      -- Looking for execinfo.h - found
      -- Looking for getline
      -- Looking for getline - found
      CUDA_TOOLKIT_ROOT_DIR not found or specified
      -- Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY)
      CMake Warning at /public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:31 (message):
        Caffe2: CUDA cannot be found.  Depending on whether you are building Caffe2
        or a Caffe2 dependent library, the next warning / error will give you more
        info.
      Call Stack (most recent call first):
        /public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:87 (include)
        /public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
        CMakeLists.txt:95 (find_package)


      CMake Error at /public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:91 (message):
        Your installed Caffe2 version uses CUDA but I cannot find the CUDA
        libraries.  Please set the proper CUDA prefixes and / or install CUDA.
      Call Stack (most recent call first):
        /public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
        CMakeLists.txt:95 (find_package)


      -- Configuring incomplete, errors occurred!
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-req-build-d6c_tul2/setup.py", line 133, in <module>
          setup(
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/__init__.py", line 104, in setup
          return distutils.core.setup(**attrs)
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 184, in setup
          return run_commands(dist)
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
          dist.run_commands()
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/dist.py", line 967, in run_command
          super().run_command(command)
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 368, in run
          self.run_command("build")
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/dist.py", line 967, in run_command
          super().run_command(command)
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 132, in run
          self.run_command(cmd_name)
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/dist.py", line 967, in run_command
          super().run_command(command)
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 91, in run
          _build_ext.run(self)
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
          self.build_extensions()
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 479, in build_extensions
          self._build_extensions_serial()
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 505, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-req-build-d6c_tul2/setup.py", line 84, in build_extension
          subprocess.check_call(['cmake', ext.sourcedir] + cmake_args,
        File "/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/subprocess.py", line 364, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['cmake', '/tmp/pip-req-build-d6c_tul2', '-DBUILD_TEST=OFF', '-DBUILD_BENCHMARK=OFF', '-DUSE_PYTHON=ON', '-DWITH_CUDA=OFF', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/tmp/pip-req-build-d6c_tul2/build/lib.linux-x86_64-cpython-38', '-DCMAKE_RUNTIME_OUTPUT_DIRECTORY=/tmp/pip-req-build-d6c_tul2/build/lib.linux-x86_64-cpython-38', '-DCMAKE_BUILD_TYPE=RELEASE', '-DCMAKE_PREFIX_PATH=/public/home/qiujiangdong/.conda/envs/jptr/lib/python3.8/site-packages/torch/share/cmake', '-GNinja']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyg_lib
  Running setup.py clean for pyg_lib
Failed to build pyg_lib
ERROR: Could not build wheels for pyg_lib, which is required to install pyproject.toml-based projects

And i supposed that we install in the login node which normally without "GPU" and carried by submit system. if this is a possible problem?

Do you have any new idea to solve this problem? Thanks

yifanfu01 avatar May 09 '24 09:05 yifanfu01

Hi @rusty1s I solved this problem, and i think it worth a detail description. I will provided in one or two days :)

yifanfu01 avatar May 10 '24 01:05 yifanfu01

Hi @yifanfu01 ,

sorry to push you. But my team really needs to solve this issue. It would be great, if you could provide us with your solution. Thanks a lot!

YufanXia999 avatar May 16 '24 09:05 YufanXia999

Hi @YufanXia999 , it's kinda difficult, and sorry to be late. I can briefly describe in advance, you should install from source rather than built version. Hence 1) download via git --clone; 2) please make sure your gcc and g++ version that i think no lower than gcc-9, and plz find the path of them; 3) use python setup.py install to install this package and please give your specific gcc path and conda path. It will not take long time.

I think, in person, this bug was caused by you install package in the login node, but it will run on the GPU node. So when installing it can not automatically decide the location of cuda. Also problems occur when find gcc, but i'm not sure whether it is same for you.

I can provide more info if any guy need : yf_pumc#163.com

Best regards, Yifan Fu

yifanfu01 avatar May 16 '24 10:05 yifanfu01

Hi @yifanfu01,

thank you so much for your quick response! I think we are having exactly the same issue. I have to install packages on login node, but jobs will run on GPU node. But the installation will fail, since on login node there are no GPU assigned.

I have tried your solution. But sadly our HPC has a version lower than gcc-9. Is there a reason why it requires gcc-9? Or do you have any other solutions? Thanks!

Best, Yufan

YufanXia999 avatar May 17 '24 15:05 YufanXia999

@YufanXia999 Hi, I think the version 7 is also enough. Could you provide your detail errors when you run python setup.py install ?

yifanfu01 avatar May 17 '24 15:05 yifanfu01

Hi @yifanfu01,

my error is same as aaronwtr's error:

CUDA_TOOLKIT_ROOT_DIR not found or specified
      -- Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY)

Basically the installation requires to detect CUDA, however, there are no GPU on login node. Do you have an idea how to solve this?

Best, Yufan

YufanXia999 avatar May 17 '24 15:05 YufanXia999

Hi @YufanXia999 I think you should specific the cuda path (might use module show YOUR/CUDA/MODULE to detect) when install by python.

yifanfu01 avatar May 17 '24 16:05 yifanfu01

Fixed by #276 in newer versions.

akihironitta avatar Jul 28 '24 15:07 akihironitta