[Bug]: undefined symbol: __nvJitLinkComplete_12_4, version libnvJitLink.so.12

[yanan1116]

trafficstars

Your current environment

i cannot execute collect_env.py because of this error.

in my another environment: torch is 2.4.0 and the version of vllm is 0.6.3.post1 which works fine.

Model Input Dumps

No response

🐛 Describe the bug

following installation guide: https://docs.vllm.ai/en/stable/getting_started/installation.html#install-the-latest-code

vllm version: 0.6.3.post2.dev386+g0b8bb86b

however, it forces the installation of torch to be 2.5.1

which causes the error :

Traceback (most recent call last): File "/home/ubuntu/vllm/collect_env.py", line 15, in from vllm.envs import environment_variables File "/home/ubuntu/vllm/vllm/init.py", line 3, in from vllm.engine.arg_utils import AsyncEngineArgs, EngineArgs File "/home/ubuntu/vllm/vllm/engine/arg_utils.py", line 8, in import torch File "/opt/conda/envs/vllmsource/lib/python3.11/site-packages/torch/init.py", line 367, in from torch._C import * # noqa: F403 ^^^^^^^^^^^^^^^^^^^^^^ ImportError: /opt/conda/envs/vllmsource/lib/python3.11/site-packages/torch/lib/../../nvidia/cusparse/lib/libcusparse.so.12: undefined symbol: __nvJitLinkComplete_12_4, version libnvJitLink.so.12

Before submitting a new issue...

• [X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[yanan1116]

related issue: https://github.com/pytorch/pytorch/issues/111469

[yanan1116]

python -c "import torch"

error:

Traceback (most recent call last): File "", line 1, in File "/opt/conda/envs/vllmsource/lib/python3.11/site-packages/torch/init.py", line 367, in from torch._C import * # noqa: F403 ^^^^^^^^^^^^^^^^^^^^^^ ImportError: /opt/conda/envs/vllmsource/lib/python3.11/site-packages/torch/lib/../../nvidia/cusparse/lib/libcusparse.so.12: undefined symbol: __nvJitLinkComplete_12_4, version libnvJitLink.so.12

[mbertrait]

I just encountered this issue, actually it seems like both torch 2.5.0 and latest 2.5.1 revive https://github.com/pytorch/pytorch/issues/111469. I opened an issue on their repo: https://github.com/pytorch/pytorch/issues/140797.

[Ecocytus]

Same issue for vllm==v0.6.4.post1 and torch==2.5.1. Cuda version 12.1.

[chintan-ushur]

Same issue, vllm==0.6.4 and torch==2.5.1

[GallonDeng]

same issue, vllm==0.6.4.post1, torch==2.5.1

[piamo]

same issue, vllm==0.6.4.post1, torch==2.5.1

[chintan-ushur]

The fix is in progress, meanwhile here is a quick workaround:

patchelf --add-rpath '$ORIGIN/../../nvidia/nvjitlink/lib' /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cuda.so Ref - https://github.com/pytorch/pytorch/issues/140797

[iseesaw]

The fix is in progress, meanwhile here is a quick workaround:

patchelf --add-rpath '$ORIGIN/../../nvidia/nvjitlink/lib' /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cuda.so Ref - https://github.com/pytorch/pytorch/issues/140797

This is very useful.

[fanfanaaaa]

it can solve the bug: export LD_LIBRARY_PATH=/data/home/user/anaconda3/envs/vllm/lib/python3.10 /site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH

[Hambaobao]

it can solve the bug: export LD_LIBRARY_PATH=/data/home/user/anaconda3/envs/vllm/lib/python3.10 /site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH

Very helpful.

[Cap-rgb]

it can solve the bug: export LD_LIBRARY_PATH=/data/home/user/anaconda3/envs/vllm/lib/python3.10 /site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH

在2025年2月8日，测试这个解决方案成功。我的情况与vllm无关，单纯使用 python -m venv test01 创建虚拟环境，然后报上面的错。使用miniconda无误。

[tchaton]

Any updates ?

[neils2019]

Any updates ?

Setting the LD_LIBRARY_PATH to search in the virtual environment directory first for the library works well, update LD_LIBRARY_PATH=<path to /nvidia/nvjitlink/lib in virtual environment>:$LD_LIBRARY_PATH

[kissint8]

it can solve the bug: export LD_LIBRARY_PATH=/data/home/user/anaconda3/envs/vllm/lib/python3.10 /site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH

2025.02.13, this method solved problem! Tks!

[MiracleLin001]

it can solve the bug: export LD_LIBRARY_PATH=/data/home/user/anaconda3/envs/vllm/lib/python3.10 /site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH

A very useful method. For pip users, the nvidia path may at /lib/python3.10/site-packages/nvidia/nvjitlink/lib

[wfloveiu]

unset LD_LIBRARY_PATH works for me!

[kuailehaha]

A very useful method. For docker users (v0.7.3), the nvidia path at /usr/local/lib/python3.12/dist-packages/nvidia/nvjitlink/lib so, export LD_LIBRARY_PATH=/usr/local/lib/python3.12/dist-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH

[kcxain]

unset LD_LIBRARY_PATH works for me!

It works, thanks!

[leslie2046]

it can solve the bug: export LD_LIBRARY_PATH=/data/home/user/anaconda3/envs/vllm/lib/python3.10 /site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH

works for me

[CauchyFanUpdate]

it can solve the bug: export LD_LIBRARY_PATH=/data/home/user/anaconda3/envs/vllm/lib/python3.10 /site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH

It is very useful !

[ZhenbinChan]

unset LD_LIBRARY_PATH works for me!

It works for me too! tks!

[Midnight-719]

it can solve the bug: export LD_LIBRARY_PATH=/data/home/user/anaconda3/envs/vllm/lib/python3.10 /site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH

A very useful method. For pip users, the nvidia path may at /lib/python3.10/site-packages/nvidia/nvjitlink/lib

thank u bro , THAT'S NICE

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[github-actions[bot]]

This issue has been automatically closed due to inactivity. Please feel free to reopen if you feel it is still relevant. Thank you!