CUDA SETUP : detection failed

[VitaLemonTea1]

Hi,

I get an error, when run another project. I think this problem is caused by bitsandbytes. And I tried the solution in other issues, but it still doesn't work. Here is my error message.

(E2WM) linjinzhou@h3c-R5300-G5:~/Project/world-model-for-language-model-main/scripts$ python -m bitsandbytes False

===================================BUG REPORT=================================== /work/linjinzhou/anaconda3/lib/python3.11/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

warn(msg)

The following directories listed in your path were found to be non-existent: {PosixPath('/work/linjinzhou/anaconda3/envs/E2WM/lib')} /work/linjinzhou/anaconda3/lib/python3.11/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: /work/linjinzhou/anaconda3/envs/E2WM did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths... warn(msg) /work/linjinzhou/anaconda3/lib/python3.11/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: /usr/local/cuda/ did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths... warn(msg) CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths... DEBUG: Possible options found for libcudart.so: {PosixPath('/usr/local/cuda/lib64/libcudart.so')} CUDA SETUP: PyTorch settings found: CUDA_VERSION=121, Highest Compute Capability: 8.6. CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md CUDA SETUP: Required library version not found: libbitsandbytes_cuda121.so. Maybe you need to compile it from source? CUDA SETUP: Defaulting to libbitsandbytes_cpu.so...

================================================ERROR===================================== CUDA SETUP: CUDA detection failed! Possible reasons:

1. You need to manually override the PyTorch CUDA version. Please see: "https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
2. CUDA driver not installed
3. CUDA not installed
4. You have multiple conflicting CUDA libraries
5. Required library not pre-compiled for this bitsandbytes release! CUDA SETUP: If you compiled from source, try again with make CUDA_VERSION=DETECTED_CUDA_VERSION for example, make CUDA_VERSION=113. CUDA SETUP: The CUDA version for the compile might depend on your conda install. Inspect CUDA version via conda list | grep cuda. ================================================================================

CUDA SETUP: Something unexpected happened. Please compile from source: git clone https://github.com/TimDettmers/bitsandbytes.git cd bitsandbytes CUDA_VERSION=121 python setup.py install CUDA SETUP: Setup Failed! Traceback (most recent call last): File "", line 189, in _run_module_as_main File "", line 148, in _get_module_details File "", line 112, in _get_module_details File "/work/linjinzhou/anaconda3/lib/python3.11/site-packages/bitsandbytes/init.py", line 6, in from . import cuda_setup, utils, research File "/work/linjinzhou/anaconda3/lib/python3.11/site-packages/bitsandbytes/research/init.py", line 1, in from . import nn File "/work/linjinzhou/anaconda3/lib/python3.11/site-packages/bitsandbytes/research/nn/init.py", line 1, in from .modules import LinearFP8Mixed, LinearFP8Global File "/work/linjinzhou/anaconda3/lib/python3.11/site-packages/bitsandbytes/research/nn/modules.py", line 8, in from bitsandbytes.optim import GlobalOptimManager File "/work/linjinzhou/anaconda3/lib/python3.11/site-packages/bitsandbytes/optim/init.py", line 6, in from bitsandbytes.cextension import COMPILED_WITH_CUDA File "/work/linjinzhou/anaconda3/lib/python3.11/site-packages/bitsandbytes/cextension.py", line 20, in raise RuntimeError(''' RuntimeError: CUDA Setup failed despite GPU being available. Please run the following command to get more information:

    python -m bitsandbytes

    Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
    to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
    and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

---

How can I solve this？

Thanks！

[younesbelkada]

This usually comes up when you have multiple CUDA versions installed on your device. Can you try to manually install bnb from source and let us know how it goes?

CUDA_VERSION=121 make cuda12x
python setup.py develop

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.