Cannot install torch-scatter and torch-sparse

[thematrixduo]

📚 Installation

Environment

• OS: Ubuntu 16.04
• Python version: 3.6.8
• PyTorch version: 1.3.0
• CUDA/cuDNN version: 10.1
• GCC version: 5.4
• How did you try to install PyTorch Geometric and its extensions (pip, source): pip install torch-scatter
• Any other relevant information:

Error message pasted below: Collecting torch-scatter Using cached https://files.pythonhosted.org/packages/30/d9/1d5fd4d183dabd9e0a1f7008ecf83318432359f4cc27480e3f2212f44d9c/torch_scatter-1.3.2.tar.gz Building wheels for collected packages: torch-scatter Building wheel for torch-scatter (setup.py) ... error ERROR: Command errored out with exit status 1: command: /local/scratch/wd263/torchEnv/bin/python3.6 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-aze98ciy/torch-scatter/setup.py'"'"'; file='"'"'/tmp/pip-install-aze98ciy/torch-scatter/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-126zvfal --python-tag cp36 cwd: /tmp/pip-install-aze98ciy/torch-scatter/ Complete output (40 lines): running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-3.6 creating build/lib.linux-x86_64-3.6/test copying test/utils.py -> build/lib.linux-x86_64-3.6/test copying test/init.py -> build/lib.linux-x86_64-3.6/test copying test/test_multi_gpu.py -> build/lib.linux-x86_64-3.6/test copying test/test_max_min.py -> build/lib.linux-x86_64-3.6/test copying test/test_std.py -> build/lib.linux-x86_64-3.6/test copying test/test_broadcasting.py -> build/lib.linux-x86_64-3.6/test copying test/test_backward.py -> build/lib.linux-x86_64-3.6/test copying test/test_forward.py -> build/lib.linux-x86_64-3.6/test creating build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/max.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/mean.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/div.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/init.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/add.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/sub.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/std.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/mul.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/min.py -> build/lib.linux-x86_64-3.6/torch_scatter creating build/lib.linux-x86_64-3.6/torch_scatter/utils copying torch_scatter/utils/init.py -> build/lib.linux-x86_64-3.6/torch_scatter/utils copying torch_scatter/utils/ext.py -> build/lib.linux-x86_64-3.6/torch_scatter/utils copying torch_scatter/utils/gen.py -> build/lib.linux-x86_64-3.6/torch_scatter/utils running build_ext building 'torch_scatter.scatter_cpu' extension creating build/temp.linux-x86_64-3.6 creating build/temp.linux-x86_64-3.6/cpu x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include -I/local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include/TH -I/local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include/THC -I/usr/include/python3.6m -I/local/scratch/wd263/torchEnv/include/python3.6m -c cpu/scatter.cpp -o build/temp.linux-x86_64-3.6/cpu/scatter.o -Wno-unused-variable -DVERSION_GE_1_3 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=scatter_cpu -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 In file included from /local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include/torch/csrc/Device.h:3:0, from /local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/python.h:8, from /local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include/torch/extension.h:6, from cpu/scatter.cpp:1: /local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include/torch/csrc/python_headers.h:9:20: fatal error: Python.h: No such file or directory compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

ERROR: Failed building wheel for torch-scatter Running setup.py clean for torch-scatter Failed to build torch-scatter Installing collected packages: torch-scatter Running setup.py install for torch-scatter ... error ERROR: Command errored out with exit status 1: command: /local/scratch/wd263/torchEnv/bin/python3.6 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-aze98ciy/torch-scatter/setup.py'"'"'; file='"'"'/tmp/pip-install-aze98ciy/torch-scatter/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-ysm4jyui/install-record.txt --single-version-externally-managed --compile --install-headers /local/scratch/wd263/torchEnv/include/site/python3.6/torch-scatter cwd: /tmp/pip-install-aze98ciy/torch-scatter/ Complete output (40 lines): running install running build running build_py creating build creating build/lib.linux-x86_64-3.6 creating build/lib.linux-x86_64-3.6/test copying test/utils.py -> build/lib.linux-x86_64-3.6/test copying test/init.py -> build/lib.linux-x86_64-3.6/test copying test/test_multi_gpu.py -> build/lib.linux-x86_64-3.6/test copying test/test_max_min.py -> build/lib.linux-x86_64-3.6/test copying test/test_std.py -> build/lib.linux-x86_64-3.6/test copying test/test_broadcasting.py -> build/lib.linux-x86_64-3.6/test copying test/test_backward.py -> build/lib.linux-x86_64-3.6/test copying test/test_forward.py -> build/lib.linux-x86_64-3.6/test creating build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/max.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/mean.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/div.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/init.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/add.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/sub.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/std.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/mul.py -> build/lib.linux-x86_64-3.6/torch_scatter copying torch_scatter/min.py -> build/lib.linux-x86_64-3.6/torch_scatter creating build/lib.linux-x86_64-3.6/torch_scatter/utils copying torch_scatter/utils/init.py -> build/lib.linux-x86_64-3.6/torch_scatter/utils copying torch_scatter/utils/ext.py -> build/lib.linux-x86_64-3.6/torch_scatter/utils copying torch_scatter/utils/gen.py -> build/lib.linux-x86_64-3.6/torch_scatter/utils running build_ext building 'torch_scatter.scatter_cpu' extension creating build/temp.linux-x86_64-3.6 creating build/temp.linux-x86_64-3.6/cpu x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include -I/local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -I/local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include/TH -I/local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include/THC -I/usr/include/python3.6m -I/local/scratch/wd263/torchEnv/include/python3.6m -c cpu/scatter.cpp -o build/temp.linux-x86_64-3.6/cpu/scatter.o -Wno-unused-variable -DVERSION_GE_1_3 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=scatter_cpu -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 In file included from /local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include/torch/csrc/Device.h:3:0, from /local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/python.h:8, from /local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include/torch/extension.h:6, from cpu/scatter.cpp:1: /local/scratch/wd263/torchEnv/lib/python3.6/site-packages/torch/include/torch/csrc/python_headers.h:9:20: fatal error: Python.h: No such file or directory compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 ---------------------------------------- ERROR: Command errored out with exit status 1: /local/scratch/wd263/torchEnv/bin/python3.6 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-aze98ciy/torch-scatter/setup.py'"'"'; file='"'"'/tmp/pip-install-aze98ciy/torch-scatter/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-ysm4jyui/install-record.txt --single-version-externally-managed --compile --install-headers /local/scratch/wd263/torchEnv/include/site/python3.6/torch-scatter Check the logs for full command output.

Additional context

[thhung]

Maybe you could try this?

https://stackoverflow.com/questions/21530577/fatal-error-python-h-no-such-file-or-directory

[rusty1s]

Yes, also mentioned here.

[thematrixduo]

I was installing these two packages in virtualenv. Interestingly when I install them instead using conda Env, there is no error.

[xxp17457741]

I was installing these two packages in virtualenv. Interestingly when I install them instead using conda Env, there is no error.

How do you install the two packages in conda Env?? I use anaconda in mac, but i do not find these two in enviroment.

[xxp17457741]

Yes, also mentioned here.

I have the same problems, also it gives me a warning

                       !! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Your compiler (g++) is not compatible with the compiler Pytorch was built with for this platform, which is clang++ on darwin. Please use clang++ to to compile your extension. Alternatively, you may compile PyTorch from source using g++, and then you can also use g++ to compile your extension.

See https://github.com/pytorch/pytorch/blob/master/CONTRIBUTING.md for help with compiling PyTorch from source. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

                            !! WARNING !!

platform=sys.platform))

building 'torch_sparse.spspmm_cpu' extension creating build/temp.macosx-10.7-x86_64-3.7 creating build/temp.macosx-10.7-x86_64-3.7/cpu gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/mac/anaconda3/include -arch x86_64 -I/Users/mac/anaconda3/include -arch x86_64 -I/Users/mac/anaconda3/lib/python3.7/site-packages/torch/include -I/Users/mac/anaconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/Users/mac/anaconda3/lib/python3.7/site-packages/torch/include/TH -I/Users/mac/anaconda3/lib/python3.7/site-packages/torch/include/THC -I/Users/mac/anaconda3/include/python3.7m -c cpu/spspmm.cpp -o build/temp.macosx-10.7-x86_64-3.7/cpu/spspmm.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=spspmm_cpu -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 dyld: Library not loaded: /usr/local/lib/libmpfr.4.dylib Referenced from: /Users/mac/usr/local/bin/../libexec/gcc/x86_64-apple-darwin19/9.2.0/cc1plus Reason: image not found gcc: internal compiler error: Abort trap: 6 signal terminated program cc1plus Please submit a full bug report, with preprocessed source if appropriate. See https://gcc.gnu.org/bugs/ for instructions. error: command 'gcc' failed with exit status 4

ERROR: Failed building wheel for torch-sparse

and in mac anaconda, it has no python-devel, I do not find the packages in conda enviroment, must use pip to install

[xxp17457741]

Yes, also mentioned here.

MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ pip install torch-scatter !!!! I get it!!!

[xxp17457741]

I was installing these two packages in virtualenv. Interestingly when I install them instead using conda Env, there is no error.

I get this!!! MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ pip install torch-scatter

[rusty1s]

Is it working for you? You need to use clang on macOS.

[xxp17457741]

Is it working for you? You need to use clang on macOS. Yes, there is a clang in my mac and i also put into a gcc[i just want pip install torch-scatter ASAP : )], but it does not work by 'pip install torch-scatter' because it is still compiled by clang. Now I find the solution and finally succeed!!! Thank you!!!

[xxp17457741]

Is it working for you? You need to use clang on macOS.

In fact, I failed finally... I tried, but still did not work, and it give me a erro. ImportError: dlopen(/Users/mac/anaconda3/lib/python3.7/site-packages/torch_scatter/scatter_cpu.cpython-37m-darwin.so, 2): Symbol not found: __ZN3c104impl28tls_excluded_tensor_type_setEv Referenced from: /Users/mac/anaconda3/lib/python3.7/site-packages/torch_scatter/scatter_cpu.cpython-37m-darwin.so Expected in: flat namespace in /Users/mac/anaconda3/lib/python3.7/site-packages/torch_scatter/scatter_cpu.cpython-37m-darwin.so

[xxp17457741]

Is it working for you? You need to use clang on macOS.

pip install torch-scatter is ok; but import torch-scatter is wrong

[xxp17457741]

Is it working for you? You need to use clang on macOS.

Anyway!! I got the 'conda install -y clang_osx-64 clangxx_osx-64 gfortran_osx-64' and succeed finally!!!!!!!!!!!!!!

[thematrixduo]

I was installing these two packages in virtualenv. Interestingly when I install them instead using conda Env, there is no error.

How do you install the two packages in conda Env?? I use anaconda in mac, but i do not find these two in enviroment.

You can just pip install them

[vipermu]

So I was having the same problem using a conda env with pytorch installde. I solved it by installing gxx_linux-64=7.3.0 and gcc_linux-64=7.3.0 .

[sanjaythakur]

My problem was solved by installing the header files and static libraries for python dev. I used the command sudo apt-get install python3-dev for it.

[fkhjoy]

My problem was solved by installing the header files and static libraries for python dev. I used the command sudo apt-get install python3-dev for it.

Mine as well