OSError: libcusparse.so.11: cannot open shared object file: No such file or directory

[turmeric-blend]

📚 Installation

---

Error occurs when I run from torch_geometric.data import Data

OSError Traceback (most recent call last) in 1 import torch ----> 2 from torch_geometric.data import Data

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_geometric/init.py in 1 from .debug import is_debug_enabled, debug, set_debug ----> 2 import torch_geometric.nn 3 import torch_geometric.data 4 import torch_geometric.datasets 5 import torch_geometric.transforms

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_geometric/nn/init.py in 1 from .meta import MetaLayer ----> 2 from .data_parallel import DataParallel 3 from .reshape import Reshape 4 from .conv import * # noqa 5 from .norm import * # noqa

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_geometric/nn/data_parallel.py in 3 4 import torch ----> 5 from torch_geometric.data import Batch 6 7

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_geometric/data/init.py in ----> 1 from .data import Data 2 from .batch import Batch 3 from .dataset import Dataset 4 from .in_memory_dataset import InMemoryDataset 5 from .dataloader import DataLoader, DataListLoader, DenseDataLoader

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_geometric/data/data.py in 6 import torch 7 import torch_geometric ----> 8 from torch_sparse import coalesce, SparseTensor 9 from torch_geometric.utils import (contains_isolated_nodes, 10 contains_self_loops, is_undirected)

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch_sparse/init.py in 11 ]: 12 torch.ops.load_library(importlib.machinery.PathFinder().find_spec( ---> 13 library, [osp.dirname(file)]).origin) 14 15 if torch.cuda.is_available() and torch.version.cuda: # pragma: no cover

~/anaconda3/envs/pytorch_env/lib/python3.7/site-packages/torch/_ops.py in load_library(self, path) 103 # static (global) initialization code in order to register custom 104 # operators with the JIT. --> 105 ctypes.CDLL(path) 106 self.loaded_libraries.add(path) 107

~/anaconda3/envs/pytorch_env/lib/python3.7/ctypes/init.py in init(self, name, mode, handle, use_errno, use_last_error) 362 363 if handle is None: --> 364 self._handle = _dlopen(self._name, mode) 365 else: 366 self._handle = handle

OSError: libcusparse.so.11: cannot open shared object file: No such file or directory

Environment

• OS: Ubuntu 18.04
• Python version: 3.7.6
• PyTorch version: 1.7.1
• CUDA/cuDNN version: 11.0
• GCC version: 7.5.0
• How did you try to install PyTorch Geometric and its extensions (wheel, source): Binaries
• Any other relevant information: followed this guide Installation via Binaries

Checklist

• [y] I followed the installation guide.
• [ ] I cannot find my error message in the FAQ.
• [ ] I set up CUDA correctly and can compile CUDA code via nvcc. <--- not sure how to do this, but I have been running pytorch on GPU fine all this while
• [ ] I do have multiple CUDA versions on my machine. <--- not sure how to check but when I run python -c "import torch; print(torch.__version__)" returns 1.7.1+cu110 and python -c "import torch; print(torch.version.cuda)" returns 11.0

Additional context

Installed via:

pip install --no-index torch-scatter -f https://pytorch-geometric.com/whl/torch-1.7.0+cu110.html
pip install --no-index torch-sparse -f https://pytorch-geometric.com/whl/torch-1.7.0+cu110.html
pip install --no-index torch-cluster -f https://pytorch-geometric.com/whl/torch-1.7.0+cu110.html
pip install --no-index torch-spline-conv -f https://pytorch-geometric.com/whl/torch-1.7.0+cu110.html
pip install torch-geometric

[rusty1s]

You can try to add the minconda CUDA libs installed by PyTorch to $LD_LIBRARY_PATH, e.g.:

export LD_LIBRARY_PATH="your_miniconda_path/lib:$LD_LIBRARY_PATH"

This folder should contain libcusparse.so.11.

[zilangch]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird.

However, I can use geometric in terminal, but can't pycharm

[rusty1s]

I'm not a PyCharm user, so I can't give you any advice on this one, but I guess PyCharm has some kind of editor to set environment variables.

[domilay]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird.

However, I can use geometric in terminal, but can't pycharm

Same problem, work well in terminal but not in pycharm...

[DeVriesMatt]

I have the same problem, however it is not working in terminal either

[rusty1s]

Please make sure that the folder containing libcusparse.so.11 is linked in LD_LIBRARY_PATH.

[DeVriesMatt]

@rusty1s I did this but still was not working. I have just got it to work through downgrading the CUDA version for both PyTorch and the PyTorch Geometric dependencies from 11.1 to 10.2

[kt66nf]

I had the exact same issue with torch 1.9.0 and cu 11.1 as mentioned in OP. I did what @DeVriesMatt did and it seems to work perfectly fine now.

[Yonggie]

I got the same problem with python3.8 torch_geometric1.9.0.

[Yonggie]

I've checked my virtual environment path your_miniconda_path/lib only to find out there's actually no file named libcusparse.so... Does that mean the torch-sparse is illedly installed?

[Yonggie]

My other virtual envs containing torch-geometric 1.8.0 does have libcusparse.so in lib, but geometric1.9.0+cu111 doesn't. though they are installed by me following exactly the same as documentation installation.

[Yonggie]

I had the exact same issue with torch 1.9.0 and cu 11.1 as mentioned in OP. I did what @DeVriesMatt did and it seems to work perfectly fine now.

I did downgrading too, solved my problem.

[tchayintr]

I had similar issue with OP.
I was set conda environment with python 3.8.x. pytorch 1.8.1 with cuda 11.1, and pyg 1.7.2. It works totally fine with cuda 11.1 but when I upgraded pytorch 1.8.1 to 1.9.0 along with compatible torch-geometric by using pip and try to import some modules e.g., GCNConv it will show the error as mentioned in OP: OSError: libcusparse.so.11: cannot open shared object file: No such file or directory.

I checked environment's lib, there actually is libcusparse.so.11 but it seems pyg could not find it somehow after upgrading. I linked the folder containing libcusparse.so.11 to LD_LIBRARY_PATH, however, it doesn't work for me.

So I tried to create new conda-environment with python 3.9.6 and install pytorch 1.9.0 with cuda11.1 plus compatible pyg by using conda. I am able to execute pyg's lib without any error so far. I could not provide an accurate reason about this issue but hope this information can help someone somehow.

[Zhang-Zhiyuan-zzy]

You can try to add the minconda CUDA libs installed by PyTorch to $LD_LIBRARY_PATH, e.g.:

export LD_LIBRARY_PATH="your_miniconda_path/lib:$LD_LIBRARY_PATH"

This folder should contain libcusparse.so.11.

@rusty1s I find that my miniconda/lib folder do not contain the "libcusparse.so.11". How can I install it?

[rusty1s]

If you are inside a custom miniconda environment, you may also find them inside miniconda3/envs/{env_name}/lib. Can you check? Note that you need to run conda install cudatoolkit=... -c pytorch first.

[kunal-bhadra]

10.2

Still works today, thanks for the info!

[tyang816]

I had similar issue with OP. I was set conda environment with python 3.8.x. pytorch 1.8.1 with cuda 11.1, and pyg 1.7.2. It works totally fine with cuda 11.1 but when I upgraded pytorch 1.8.1 to 1.9.0 along with compatible torch-geometric by using pip and try to import some modules e.g., GCNConv it will show the error as mentioned in OP: OSError: libcusparse.so.11: cannot open shared object file: No such file or directory.

I checked environment's lib, there actually is libcusparse.so.11 but it seems pyg could not find it somehow after upgrading. I linked the folder containing libcusparse.so.11 to LD_LIBRARY_PATH, however, it doesn't work for me.

So I tried to create new conda-environment with python 3.9.6 and install pytorch 1.9.0 with cuda11.1 plus compatible pyg by using conda. I am able to execute pyg's lib without any error so far. I could not provide an accurate reason about this issue but hope this information can help someone somehow.

Thanks! conda install is useful.

[smiles724]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird.

However, I can use geometric in terminal, but can't pycharm

I met a similar case. I can run it in the terminal but not in the jupyter notebook

[rusty1s]

can you add the path inside the jupyter notebook as well?

[MirMurtazaa]

can you add the path inside the jupyter notebook as well?

I added the path in Jupyter Notebook, but did not work. Any suggestions? Thanks

[MirMurtazaa]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird. However, I can use geometric in terminal, but can't pycharm

I met a similar case. I can run it in the terminal but not in the jupyter notebook

I solved this by:

Steps:

1. Open the .bashrc file in your home directory (for example, /home/your-user-name/.bashrc) in a text editor.
2. Add export "export LD_LIBRARY_PATH="your_minicondaORanaconda_path/lib:$LD_LIBRARY_PATH"" to the last line of the file
3. Save the .bashrc file.
4. Restart(maybe)

[chinggg]

For those who cannot find libcusparse.so in anywhere and don't have root permission to install it easily, check out https://github.com/pyg-team/pytorch_geometric/issues/392#issuecomment-503335625 to install CUDA toolkit without root.

[HelloWorldLTY]

Hi, I meet this problem again. libcusparse.so.11: cannot open shared object file: No such file or directory

I tried to use conda install to install pyg. Does anyone successfully do that? Thanks.

[rusty1s]

What is your PyTorch version and how did you install it? Do you have a local CUDA version installed?

[xuhongzuo]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird.

However, I can use geometric in terminal, but can't pycharm

@smiles724 @zilangch @domilay I found a solution in Pycharm. It can be solved by manually adding LIBRARY_PATH in the run/debug configurations.

[gorkamunoz]

I had similar issue with OP. I was set conda environment with python 3.8.x. pytorch 1.8.1 with cuda 11.1, and pyg 1.7.2. It works totally fine with cuda 11.1 but when I upgraded pytorch 1.8.1 to 1.9.0 along with compatible torch-geometric by using pip and try to import some modules e.g., GCNConv it will show the error as mentioned in OP: OSError: libcusparse.so.11: cannot open shared object file: No such file or directory.

I checked environment's lib, there actually is libcusparse.so.11 but it seems pyg could not find it somehow after upgrading. I linked the folder containing libcusparse.so.11 to LD_LIBRARY_PATH, however, it doesn't work for me.

So I tried to create new conda-environment with python 3.9.6 and install pytorch 1.9.0 with cuda11.1 plus compatible pyg by using conda. I am able to execute pyg's lib without any error so far. I could not provide an accurate reason about this issue but hope this information can help someone somehow.

Using conda install also solved my problem. In my case I was facing the same problem but with CUDA toolkit 11.6.

[zhuwangjulia]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird.

However, I can use geometric in terminal, but can't pycharm

if the above method does not work, you can try to add the following code in your main.py import sys sys.path.append('/home/name/.conda/envs/envs_name/') Also the path should contain the file libcusparse.so.11 .

[kietbg0079]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird. However, I can use geometric in terminal, but can't pycharm

I met a similar case. I can run it in the terminal but not in the jupyter notebook

I solved this by:

Steps:

1. Open the .bashrc file in your home directory (for example, /home/your-user-name/.bashrc) in a text editor.
2. Add export "export LD_LIBRARY_PATH="your_minicondaORanaconda_path/lib:$LD_LIBRARY_PATH"" to the last line of the file
3. Save the .bashrc file.
4. Restart(maybe)

I met the same issue, it can run in the terminal but failed when run in jupyter. But this solution doesn't work for me. P

[FancyRay]

I have write it "export PATH=/usr/local/cuda-11.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH" in ~/.bashrc, but it doesn't work, it's so weird. However, I can use geometric in terminal, but can't pycharm

@smiles724 @zilangch @domilay I found a solution in Pycharm. It can be solved by manually adding LIBRARY_PATH in the run/debug configurations.

Hi, thanks for your solution. But i wonder wether i have to manually edit the run/debug configurations everytime i run a new python script. I try to add the environment variables in setting-Build,Excution,Deployment-Console in pycharm, but it did not work. Did you find any solution for this?

[harshap-ai]

Check if nvcc --version works. If not, do sudo apt install nvidia-cuda-tookit.

Even if you had lubcusparse.so.11 in your lib directory prior to this step, it maybe outdated/broken. Doing the above install worked for me.