bitsandbytes icon indicating copy to clipboard operation
bitsandbytes copied to clipboard
verified publisher icon TimDettmers

CUDA setup failed

Open Will-Zhao0 opened this issue 1 year ago • 5 comments

System Info

ubuntu 20.04 A100 CUDA version 12.2

+---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.104.12 Driver Version: 535.104.12 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================|

Reproduction

python -m bitsandbytes

Expected behavior

False

===================================BUG REPORT=================================== /home/wz1937/miniconda3/envs/learn/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:167: UserWarning: Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

warn(msg)

/home/wz1937/miniconda3/envs/learn/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:167: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/home/wz1937/miniconda3/envs/learn/lib/libcudart.so.11.0'), PosixPath('/home/wz1937/miniconda3/envs/learn/lib/libcudart.so')}.. We select the PyTorch default libcudart.so, which is {torch.version.cuda},but this might missmatch with the CUDA version that is needed for bitsandbytes.To override this behavior set the BNB_CUDA_VERSION=<version string, e.g. 122> environmental variableFor example, if you want to use the CUDA version 122BNB_CUDA_VERSION=122 python ...OR set the environmental variable in your .bashrc: export BNB_CUDA_VERSION=122In the case of a manual override, make sure you set the LD_LIBRARY_PATH, e.g.export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.2 warn(msg) /home/wz1937/miniconda3/envs/learn/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:167: UserWarning: /home/wz1937/miniconda3/envs/learn did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths... warn(msg) /home/wz1937/miniconda3/envs/learn/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:167: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('miniconda3/envs/learn/lib/libcudart.so'), PosixPath('miniconda3/envs/learn/lib/libcudart.so.11.0')}.. We select the PyTorch default libcudart.so, which is {torch.version.cuda},but this might missmatch with the CUDA version that is needed for bitsandbytes.To override this behavior set the BNB_CUDA_VERSION=<version string, e.g. 122> environmental variableFor example, if you want to use the CUDA version 122BNB_CUDA_VERSION=122 python ...OR set the environmental variable in your .bashrc: export BNB_CUDA_VERSION=122In the case of a manual override, make sure you set the LD_LIBRARY_PATH, e.g.export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.2 warn(msg) /home/wz1937/miniconda3/envs/learn/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:167: UserWarning: miniconda3/envs/learn/lib/:/usr/local/lib:/usr/local/lib: did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths... warn(msg) The following directories listed in your path were found to be non-existent: {PosixPath('file'), PosixPath('/home/wz1937/miniconda3/envs/learn/etc/xml/catalog file')} CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths... DEBUG: Possible options found for libcudart.so: {PosixPath('/usr/local/cuda/lib64/libcudart.so')} CUDA SETUP: PyTorch settings found: CUDA_VERSION=121, Highest Compute Capability: 7.5. CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md CUDA SETUP: Required library version not found: libbitsandbytes_cuda121.so. Maybe you need to compile it from source? CUDA SETUP: Defaulting to libbitsandbytes_cpu.so...

================================================ERROR===================================== CUDA SETUP: CUDA detection failed! Possible reasons:

  1. You need to manually override the PyTorch CUDA version. Please see: "https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
  2. CUDA driver not installed
  3. CUDA not installed
  4. You have multiple conflicting CUDA libraries
  5. Required library not pre-compiled for this bitsandbytes release! CUDA SETUP: If you compiled from source, try again with make CUDA_VERSION=DETECTED_CUDA_VERSION for example, make CUDA_VERSION=113. CUDA SETUP: The CUDA version for the compile might depend on your conda install. Inspect CUDA version via conda list | grep cuda. ================================================================================

CUDA SETUP: Something unexpected happened. Please compile from source: git clone https://github.com/TimDettmers/bitsandbytes.git cd bitsandbytes CUDA_VERSION=121 python setup.py install CUDA SETUP: Setup Failed! Traceback (most recent call last): File "/home/wz1937/miniconda3/envs/learn/lib/python3.9/runpy.py", line 188, in _run_module_as_main mod_name, mod_spec, code = _get_module_details(mod_name, _Error) File "/home/wz1937/miniconda3/envs/learn/lib/python3.9/runpy.py", line 147, in _get_module_details return _get_module_details(pkg_main_name, error) File "/home/wz1937/miniconda3/envs/learn/lib/python3.9/runpy.py", line 111, in _get_module_details import(pkg_name) File "/home/wz1937/miniconda3/envs/learn/lib/python3.9/site-packages/bitsandbytes/init.py", line 6, in from . import cuda_setup, utils, research File "/home/wz1937/miniconda3/envs/learn/lib/python3.9/site-packages/bitsandbytes/research/init.py", line 1, in from . import nn File "/home/wz1937/miniconda3/envs/learn/lib/python3.9/site-packages/bitsandbytes/research/nn/init.py", line 1, in from .modules import LinearFP8Mixed, LinearFP8Global File "/home/wz1937/miniconda3/envs/learn/lib/python3.9/site-packages/bitsandbytes/research/nn/modules.py", line 8, in from bitsandbytes.optim import GlobalOptimManager File "/home/wz1937/miniconda3/envs/learn/lib/python3.9/site-packages/bitsandbytes/optim/init.py", line 6, in from bitsandbytes.cextension import COMPILED_WITH_CUDA File "/home/wz1937/miniconda3/envs/learn/lib/python3.9/site-packages/bitsandbytes/cextension.py", line 20, in raise RuntimeError(''' RuntimeError: CUDA Setup failed despite GPU being available. Please run the following command to get more information:

    python -m bitsandbytes

    Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
    to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
    and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

Will-Zhao0 avatar Feb 19 '24 05:02 Will-Zhao0

Hi I've encountered the same problem in Google Colab. Have you found any solutions?

FTaheriN avatar Feb 23 '24 06:02 FTaheriN

not a definite solution, but there is a specific version of cuda libcudart.so bnb requires, so I did a find -name libbitsandbytes*.so, and I did corresponding manual setting cuda version mentioned in their docs.

Will-Zhao0 avatar Feb 29 '24 17:02 Will-Zhao0

Hi. I am facing the Exact problem can anyone provide the exact solution?

===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

bin c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\libbitsandbytes_cuda123.dll False CUDA SETUP: CUDA runtime path found: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.3\bin\cudart64_12.dll CUDA SETUP: Highest compute capability among GPUs detected: 8.6 CUDA SETUP: Detected CUDA version 123 CUDA SETUP: Required library version not found: libbitsandbytes_cuda123.dll. Maybe you need to compile it from source? CUDA SETUP: Defaulting to libbitsandbytes_cpu.dll...

================================================ERROR===================================== CUDA SETUP: CUDA detection failed! Possible reasons:

  1. CUDA driver not installed
  2. CUDA not installed
  3. You have multiple conflicting CUDA libraries
  4. Required library not pre-compiled for this bitsandbytes release! CUDA SETUP: If you compiled from source, try again with make CUDA_VERSION=DETECTED_CUDA_VERSION for example, make CUDA_VERSION=113. CUDA SETUP: The CUDA version for the compile might depend on your conda install. Inspect CUDA version via conda list | grep cuda. ================================================================================

CUDA SETUP: Something unexpected happened. Please compile from source: git clone [email protected]:TimDettmers/bitsandbytes.git cd bitsandbytes CUDA_VERSION=123 python setup.py install CUDA SETUP: Setup Failed! c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\cuda_setup\main.py:156: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('C:/Users/mustf/anaconda3/envs/py310/bin')} warn(msg) c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\cuda_setup\main.py:156: UserWarning: C:\Users\mustf\anaconda3\envs\py310 did not contain ['cudart64_110.dll', 'cudart64_120.dll', 'cudart64_12.dll'] as expected! Searching further paths... warn(msg) Traceback (most recent call last): File "c:\Users\mustf\anaconda3\envs\py310\lib\runpy.py", line 187, in _run_module_as_main mod_name, mod_spec, code = _get_module_details(mod_name, Error) File "c:\Users\mustf\anaconda3\envs\py310\lib\runpy.py", line 146, in get_module_details return get_module_details(pkg_main_name, error) File "c:\Users\mustf\anaconda3\envs\py310\lib\runpy.py", line 110, in get_module_details import(pkg_name) File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes_init.py", line 6, in from . import cuda_setup, utils, research File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\research_init.py", line 1, in from . import nn File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\research\nn_init.py", line 1, in from .modules import LinearFP8Mixed, LinearFP8Global File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\research\nn\modules.py", line 8, in from bitsandbytes.optim import GlobalOptimManager File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\optim_init.py", line 6, in from bitsandbytes.cextension import COMPILED_WITH_CUDA File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\cextension.py", line 20, in raise RuntimeError(''' RuntimeError: CUDA Setup failed despite GPU being available. Please run the following command to get more information:

    python -m bitsandbytes

    Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
    to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
    and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

MustfainTariq avatar Mar 04 '24 22:03 MustfainTariq

Hi. I am facing the Exact problem can anyone provide the exact solution?

===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

bin c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\libbitsandbytes_cuda123.dll False CUDA SETUP: CUDA runtime path found: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.3\bin\cudart64_12.dll CUDA SETUP: Highest compute capability among GPUs detected: 8.6 CUDA SETUP: Detected CUDA version 123 CUDA SETUP: Required library version not found: libbitsandbytes_cuda123.dll. Maybe you need to compile it from source? CUDA SETUP: Defaulting to libbitsandbytes_cpu.dll...

================================================ERROR===================================== CUDA SETUP: CUDA detection failed! Possible reasons:

  1. CUDA driver not installed
  2. CUDA not installed
  3. You have multiple conflicting CUDA libraries

  4. Required library not pre-compiled for this bitsandbytes release! CUDA SETUP: If you compiled from source, try again with make CUDA_VERSION=DETECTED_CUDA_VERSION for example, make CUDA_VERSION=113. CUDA SETUP: The CUDA version for the compile might depend on your conda install. Inspect CUDA version via conda list | grep cuda.

CUDA SETUP: Something unexpected happened. Please compile from source: git clone [email protected]:TimDettmers/bitsandbytes.git cd bitsandbytes CUDA_VERSION=123 python setup.py install CUDA SETUP: Setup Failed! c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\cuda_setup\main.py:156: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('C:/Users/mustf/anaconda3/envs/py310/bin')} warn(msg) c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\cuda_setup\main.py:156: UserWarning: C:\Users\mustf\anaconda3\envs\py310 did not contain ['cudart64_110.dll', 'cudart64_120.dll', 'cudart64_12.dll'] as expected! Searching further paths... warn(msg) Traceback (most recent call last): File "c:\Users\mustf\anaconda3\envs\py310\lib\runpy.py", line 187, in run_module_as_main mod_name, mod_spec, code = get_module_details(mod_name, Error) File "c:\Users\mustf\anaconda3\envs\py310\lib\runpy.py", line 146, in get_module_details return get_module_details(pkg_main_name, error) File "c:\Users\mustf\anaconda3\envs\py310\lib\runpy.py", line 110, in get_module_details import(pkg_name) File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes__init.py", line 6, in from . import cuda_setup, utils, research File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\research__init.py", line 1, in from . import nn File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\research\nn__init.py", line 1, in from .modules import LinearFP8Mixed, LinearFP8Global File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\research\nn\modules.py", line 8, in from bitsandbytes.optim import GlobalOptimManager File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\optim__init.py", line 6, in from bitsandbytes.cextension import COMPILED_WITH_CUDA File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\cextension.py", line 20, in raise RuntimeError(''' RuntimeError: CUDA Setup failed despite GPU being available. Please run the following command to get more information:__

    python -m bitsandbytes

    Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
    to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
    and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

Hi The following versions worked fine for me. Hope it helps. !pip install -q accelerate==0.21.0 peft==0.4.0 bitsandbytes==0.40.2 transformers==4.31.0 trl==0.4.7

FTaheriN avatar Mar 05 '24 09:03 FTaheriN

Hi. I am facing the Exact problem can anyone provide the exact solution? ===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please run python -m bitsandbytes

and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

bin c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\libbitsandbytes_cuda123.dll False CUDA SETUP: CUDA runtime path found: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.3\bin\cudart64_12.dll CUDA SETUP: Highest compute capability among GPUs detected: 8.6 CUDA SETUP: Detected CUDA version 123 CUDA SETUP: Required library version not found: libbitsandbytes_cuda123.dll. Maybe you need to compile it from source? CUDA SETUP: Defaulting to libbitsandbytes_cpu.dll... ================================================ERROR===================================== CUDA SETUP: CUDA detection failed! Possible reasons:

  1. CUDA driver not installed
  2. CUDA not installed
  3. You have multiple conflicting CUDA libraries

  4. Required library not pre-compiled for this bitsandbytes release!

    CUDA SETUP: If you compiled from source, try again with make CUDA_VERSION=DETECTED_CUDA_VERSION for example, make CUDA_VERSION=113. CUDA SETUP: The CUDA version for the compile might depend on your conda install. Inspect CUDA version via conda list | grep cuda.

CUDA SETUP: Something unexpected happened. Please compile from source: git clone [email protected]:TimDettmers/bitsandbytes.git cd bitsandbytes CUDA_VERSION=123 python setup.py install CUDA SETUP: Setup Failed! c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\cuda_setup\main.py:156: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('C:/Users/mustf/anaconda3/envs/py310/bin')} warn(msg) c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\cuda_setup\main.py:156: UserWarning: C:\Users\mustf\anaconda3\envs\py310 did not contain ['cudart64_110.dll', 'cudart64_120.dll', 'cudart64_12.dll'] as expected! Searching further paths... warn(msg) Traceback (most recent call last): File "c:\Users\mustf\anaconda3\envs\py310\lib\runpy.py", line 187, in run_module_as_main mod_name, mod_spec, code = get_module_details(mod_name, Error) File "c:\Users\mustf\anaconda3\envs\py310\lib\runpy.py", line 146, in get_module_details return get_module_details(pkg_main_name, error) File "c:\Users\mustf\anaconda3\envs\py310\lib\runpy.py", line 110, in get_module_details import(pkg_name) File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes__init.py", line 6, in from . import cuda_setup, utils, research File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\research__init.py", line 1, in from . import nn File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\research\nn__init.py", line 1, in from .modules import LinearFP8Mixed, LinearFP8Global File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\research\nn\modules.py", line 8, in from bitsandbytes.optim import GlobalOptimManager File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\optim__init.py", line 6, in from bitsandbytes.cextension import COMPILED_WITH_CUDA File "c:\Users\mustf\anaconda3\envs\py310\lib\site-packages\bitsandbytes\cextension.py", line 20, in raise RuntimeError(''' RuntimeError: CUDA Setup failed despite GPU being available. Please run the following command to get more information:__

    python -m bitsandbytes

    Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
    to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
    and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

Hi The following versions worked fine for me. Hope it helps. !pip install -q accelerate==0.21.0 peft==0.4.0 bitsandbytes==0.40.2 transformers==4.31.0 trl==0.4.7

@FTaheriN thanks for your answer. I'm using colab and it works for me

BlingDan avatar Mar 20 '24 08:03 BlingDan