bitsandbytes
bitsandbytes copied to clipboard
TimDettmers
Dreambooth sd-webui: CUDA Setup failed despite GPU being available.
System Info
Google Cloud Platform, Deeplearning VM image Debian GNU/Linux 11 (bullseye) Build cuda_12.2.r12.2/compiler.33191640_0 NVIDIA-SMI 525.125.06 Driver Version: 525.125.06 CUDA Version: 12.0 python 3.10 [+] torch version 2.0.1+cu118 installed. [+] torchvision version 0.15.2+cu118 installed. [+] accelerate version 0.21.0 installed. [+] diffusers version 0.25.1 installed. [+] bitsandbytes version 0.41.2.post2 installed. [!] xformers version 0.0.20 installed.
Reproduction
Set up Deeplearning VM on GCP Install sd-webui Launch Web UI with arguments: --xformers --listen --no-half-vae --enable-insecure-extension-access Install Dreambooth through Extensions tab restart sd-webui
redacted console output: (venv) (base) XXXXXXXXXXXXXXX@usc1-spot-sdwebui:~/sd-webui/stable-diffusion-webui$ python -m bitsandbytes False
===================================BUG REPORT=================================== /home/XXXXXXXXXXXXXXX/sd-webui/stable-diffusion-webui/venv/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
warn(msg)
/home/XXXXXXXXXXXXXXX/sd-webui/stable-diffusion-webui/venv/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: /opt/conda did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
/home/XXXXXXXXXXXXXXX/sd-webui/stable-diffusion-webui/venv/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: :/opt/conda/lib:/opt/conda/lib did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
The following directories listed in your path were found to be non-existent: {PosixPath('-DCMAKE_AR=/opt/conda/bin/x86_64-conda-linux-gnu-ar -DCMAKE_CXX_COMPILER_AR=/opt/conda/bin/x86_64-conda-linux-gnu-gcc-ar -DCMAKE_C_COMPILER_AR=/opt/conda/bin/x86_64-conda-linux-gnu-gcc-ar -DCMAKE_RANLIB=/opt/conda/bin/x86_64-conda-linux-gnu-ranlib -DCMAKE_CXX_COMPILER_RANLIB=/opt/conda/bin/x86_64-conda-linux-gnu-gcc-ranlib -DCMAKE_C_COMPILER_RANLIB=/opt/conda/bin/x86_64-conda-linux-gnu-gcc-ranlib -DCMAKE_LINKER=/opt/conda/bin/x86_64-conda-linux-gnu-ld -DCMAKE_STRIP=/opt/conda/bin/x86_64-conda-linux-gnu-strip -DCMAKE_BUILD_TYPE=Release')}
The following directories listed in your path were found to be non-existent: {PosixPath('/opt/c2d/post_start.sh')}
The following directories listed in your path were found to be non-existent: {PosixPath('file'), PosixPath('/opt/conda/etc/xml/catalog file'), PosixPath('/etc/xml/catalog')}
The following directories listed in your path were found to be non-existent: {PosixPath('/proxy/%PORT%')}
The following directories listed in your path were found to be non-existent: {PosixPath('-fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /opt/conda/include -I/opt/conda/targets/x86_64-linux/include -L/opt/conda/targets/x86_64-linux/lib -L/opt/conda/targets/x86_64-linux/lib/stubs')}
The following directories listed in your path were found to be non-existent: {PosixPath('-fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -ffunction-sections -pipe -isystem /opt/conda/include')}
The following directories listed in your path were found to be non-existent: {PosixPath('-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/opt/conda/lib -Wl,-rpath-link,/opt/conda/lib -L/opt/conda/lib -L/opt/conda/targets/x86_64-linux/lib -L/opt/conda/targets/x86_64-linux/lib/stubs')}
The following directories listed in your path were found to be non-existent: {PosixPath('-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -ffunction-sections -pipe -isystem /opt/conda/include')}
The following directories listed in your path were found to be non-existent: {PosixPath(' -ccbin=/opt/conda/bin/x86_64-conda-linux-gnu-c++ -ccbin=/opt/conda/bin/x86_64-conda-linux-gnu-c++')}
The following directories listed in your path were found to be non-existent: {PosixPath('-DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /opt/conda/include -I/opt/conda/targets/x86_64-linux/include -L/opt/conda/targets/x86_64-linux/lib -L/opt/conda/targets/x86_64-linux/lib/stubs')}
The following directories listed in your path were found to be non-existent: {PosixPath('/proxy/%PORT%')}
The following directories listed in your path were found to be non-existent: {PosixPath('-D_DEBUG -D_FORTIFY_SOURCE=2 -Og -isystem /opt/conda/include')}
The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/share/ca-certificates,/etc/ssl/certs,/etc/ca-certificates/update.d')}
The following directories listed in your path were found to be non-existent: {PosixPath(' -ccbin=/opt/conda/bin/x86_64-conda-linux-gnu-c++')}
The following directories listed in your path were found to be non-existent: {PosixPath('-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /opt/conda/include -I/opt/conda/targets/x86_64-linux/include -L/opt/conda/targets/x86_64-linux/lib -L/opt/conda/targets/x86_64-linux/lib/stubs')}
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/cuda/lib64')}
DEBUG: Possible options found for libcudart.so: set()
CUDA SETUP: PyTorch settings found: CUDA_VERSION=118, Highest Compute Capability: 7.5.
CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
CUDA SETUP: Loading binary /home/XXXXXXXXXXXXXXX/sd-webui/stable-diffusion-webui/venv/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda118.so...
libcusparse.so.11: cannot open shared object file: No such file or directory
CUDA SETUP: Problem: The main issue seems to be that the main CUDA runtime library was not detected.
CUDA SETUP: Solution 1: To solve the issue the libcudart.so location needs to be added to the LD_LIBRARY_PATH variable
CUDA SETUP: Solution 1a): Find the cuda runtime library via: find / -name libcudart.so 2>/dev/null
CUDA SETUP: Solution 1b): Once the library is found add it to the LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:FOUND_PATH_FROM_1a
CUDA SETUP: Solution 1c): For a permanent solution add the export from 1b into your .bashrc file, located at ~/.bashrc
CUDA SETUP: Solution 2: If no library was found in step 1a) you need to install CUDA.
CUDA SETUP: Solution 2a): Download CUDA install script: wget https://github.com/TimDettmers/bitsandbytes/blob/main/cuda_install.sh
CUDA SETUP: Solution 2b): Install desired CUDA version to desired location. The syntax is bash cuda_install.sh CUDA_VERSION PATH_TO_INSTALL_INTO.
CUDA SETUP: Solution 2b): For example, "bash cuda_install.sh 113 ~/local/" will download CUDA 11.3 and install into the folder ~/local
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/runpy.py", line 187, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/opt/conda/lib/python3.10/runpy.py", line 146, in _get_module_details
return _get_module_details(pkg_main_name, error)
File "/opt/conda/lib/python3.10/runpy.py", line 110, in _get_module_details
import(pkg_name)
File "/home/XXXXXXXXXXXXXXX/sd-webui/stable-diffusion-webui/venv/lib/python3.10/site-packages/bitsandbytes/init.py", line 6, in
python -m bitsandbytes
Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues
Expected behavior
sd-webui should launch and function as it did before installation
CUDA_VERSION=118 make cuda11x
I read this command here and there but I don't understand, you can't put that in a terminal (in windows in any case)
I guess you have to
CUDA_VERSION=118 make cuda11x python setup.py install
Why 118? Their output clearly says CUDA Version 12.0
@mark20044 I guess @Entretoize just meant to compile and install from source, like this suggested answer: https://github.com/TimDettmers/bitsandbytes/issues/551#issuecomment-1621292363
This link help me resolve problems with dependencies. Reinstalling torchvision, torch and xformers python packages solved problem.
As someone mentioned Windows here, I'm posting the below:
Dear all,
Since the current release (last week, 8th of March) we now have official support for Windows 🎉 (which we did not have before) via
pip install bitsandbytes>=0.43.0
We're closing all old Windows issues and are asking everyone to try installing with this new version as outlined above and validate the install with python -m bitsandbytes which should spit out a bunch of stuff and then SUCCESS. Please let us know if everything worked correctly in this new umbrella / catch-all issue. Thanks 🤗
As to the other concerns, we're sorry that the debug messages are a bit hard to grok for many as they're a bit confusing. Since taking over maintenance recently, we've identified this issue and are working hard on a solution to make diagnostics clearer. This should already be greatly improved in the next two releases. However, what doesn't come across is that usually the issue is not with bitsandbytes itself, but with the underlying CUDA installation.
Anyways, on a side-note, we've also changed our compilation backend and you now have to follow these instructions when compiling from source.
Please let us know if any of you still experience issues, otherwise I'll close this issues sometime after 3 weeks from now.
Thanks everyone for your understanding. We're working hard to make your experience better 🤗
