bitsandbytes
bitsandbytes copied to clipboard
TimDettmers
error message not meaningful enough — why is it looking in folders that are not in my path and are unrelated environment variables?
System Info
Windows 10 [Version 10.0.19045.4529] Python 3.10.9 | packaged by Anaconda, Inc. | (main, Mar 1 2023, 18:18:15) [MSC v.1916 64 bit (AMD64)] on win32 RTX3060 12G
nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2023 NVIDIA Corporation Built on Mon_Apr__3_17:36:15_Pacific_Daylight_Time_2023 Cuda compilation tools, release 12.1, V12.1.105
Reproduction
First off — it seems to have a lot of errors related to my path, which leak values from my environment that have absolutely nothing to do with this. I don't know if environment variables are being tread upon in an untoward way or what is going on at all.
So here is my path, which is rather unremarkable to me:
C:\>path PATH=C:\BAT\DEMONA;C:\BAT\;C:\UTIL\;"c:\Program Files\ImageMagick";c:\perl\perl\site\bin;c:\perl\perl\bin;C:\perl\c\bin;c:\util\aspell\bin;c:\cygwin\bin;c:\cygwin64\bin;c:\MinGW\bin;C:\bat\work;c:\perl\bin;C:\UTIL\sysinternals;C:\UTIL2\git\bin;C:\UTIL2;C:\UTIL2\nmap;C:\UTIL2\emulation\xbox;"C:\Program Files (x86)\Common Files\Oracle\Java\javapath";C:\ProgramData\anaconda3\condabin\;C:\ProgramData\anaconda3\Scripts\;C:\ProgramData\anaconda3\Library\bin\;C:\ProgramData\miniconda3\condabin\;C:\ProgramData\miniconda3\Scripts\;C:\ProgramData\miniconda3\Library\bin\;"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\bin";"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\lib\x64";"C:\Program Files\Graphviz\bin";C:\Users\ClioC\.cargo\bin;C:\ProgramData\chocolatey\bin\;"C:\Program Files\Docker\cli-plugins";"C:\Program Files\Docker\Docker\resources\bin";"C:\ProgramData\anaconda3\Lib\site-packages\torch\lib";C:\Windows;C:\Windows\system32;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH;c:\TCMD;C:\Windows\system32\Wbem;C:\BAT
Here is the command I am running [for what it's worth, I'm in the output_dir specified, when running this]:
whisperx.exe --verbose True --language en --threads 12 --device cuda --model large-v2 --output_dir "C:\new\MUSIC\changerrecent\Destroy Boys\2016 - Sorry, Mom" --output_format lrc --beam_size 5 "1_K Street Walker.vocals.wav"
But I get this monster error output that I can't grok:
``
===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
bin C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('call winamp /add')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/e/j')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('%ANSI_RED%%ESCAPE( @./%ESCAPE%(B%ANSI_RESET%')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('%ESCAPE( @./%ESCAPE%(B')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('http'), WindowsPath('/n6905.oipcam.com'), WindowsPath('80')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('//192.168.0.151/main.htm/'), WindowsPath('http')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/n5262.oipcam.com'), WindowsPath('http'), WindowsPath('80')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('//192.168.0.121/main.htm/'), WindowsPath('http')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('T'), WindowsPath('/mp3-svn-repo')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('file'), WindowsPath('/T'), WindowsPath('/mp3-svn-repo')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/localhost'), WindowsPath('http'), WindowsPath('666/main')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/claire.sheer.us'), WindowsPath('http')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/TRANS')}
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/usr/local/cuda/lib64')}
warn(msg)
CUDA SETUP: WARNING! libcuda.so not found! Do you have a CUDA driver installed? If you are on a cluster, make sure you are on a CUDA machine!
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)!
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: No GPU detected! Check your CUDA paths. Proceeding to load CPU-only library...
warn(msg)
CUDA SETUP: Loading binary C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so...
argument of type 'WindowsPath' is not iterable
CUDA SETUP: Problem: The main issue seems to be that the main CUDA library was not detected.
CUDA SETUP: Solution 1): Your paths are probably not up-to-date. You can update them via: sudo ldconfig.
CUDA SETUP: Solution 2): If you do not have sudo rights, you can do the following:
CUDA SETUP: Solution 2a): Find the cuda library via: find / -name libcuda.so 2>/dev/null
CUDA SETUP: Solution 2b): Once the library is found add it to the LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:FOUND_PATH_FROM_2a
CUDA SETUP: Solution 2c): For a permanent solution add the export from 2b into your .bashrc file, located at ~/.bashrc
Traceback (most recent call last):
File "C:\ProgramData\anaconda3\lib\site-packages\transformers\utils\import_utils.py", line 1086, in get_module
return importlib.import_module("." + module_name, self.name)
File "C:\ProgramData\anaconda3\lib\importlib_init.py", line 126, in import_module
return _bootstrap.gcd_import(name[level:], package, level)
File "
python -m bitsandbytes
Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\ProgramData\anaconda3\lib\runpy.py", line 196, in _run_module_as_main
return run_code(code, main_globals, None,
File "C:\ProgramData\anaconda3\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\ProgramData\anaconda3\Scripts\whisperx.exe_main.py", line 4, in
CUDA Setup failed despite GPU being available. Please run the following command to get more information:
python -m bitsandbytes
Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues
``
And since it asks me to run the python -m bitsandbytes command, here's that output"
`` python -m bitsandbytes
===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
bin C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('call winamp /add')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/e/j')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('%ANSI_RED%%ESCAPE( @./%ESCAPE%(B%ANSI_RESET%')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('%ESCAPE( @./%ESCAPE%(B')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/n6905.oipcam.com'), WindowsPath('80'), WindowsPath('http')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('//192.168.0.151/main.htm/'), WindowsPath('http')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/n5262.oipcam.com'), WindowsPath('80'), WindowsPath('http')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('//192.168.0.121/main.htm/'), WindowsPath('http')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/mp3-svn-repo'), WindowsPath('T')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/mp3-svn-repo'), WindowsPath('/T'), WindowsPath('file')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/localhost'), WindowsPath('666/main'), WindowsPath('http')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/claire.sheer.us'), WindowsPath('http')}
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/TRANS')}
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('/usr/local/cuda/lib64')}
warn(msg)
CUDA SETUP: WARNING! libcuda.so not found! Do you have a CUDA driver installed? If you are on a cluster, make sure you are on a CUDA machine!
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)!
warn(msg)
C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\cuda_setup\main.py:145: UserWarning: WARNING: No GPU detected! Check your CUDA paths. Proceeding to load CPU-only library...
warn(msg)
CUDA SETUP: Loading binary C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.so...
argument of type 'WindowsPath' is not iterable
CUDA SETUP: Problem: The main issue seems to be that the main CUDA library was not detected.
CUDA SETUP: Solution 1): Your paths are probably not up-to-date. You can update them via: sudo ldconfig.
CUDA SETUP: Solution 2): If you do not have sudo rights, you can do the following:
CUDA SETUP: Solution 2a): Find the cuda library via: find / -name libcuda.so 2>/dev/null
CUDA SETUP: Solution 2b): Once the library is found add it to the LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:FOUND_PATH_FROM_2a
CUDA SETUP: Solution 2c): For a permanent solution add the export from 2b into your .bashrc file, located at ~/.bashrc
Traceback (most recent call last):
File "C:\ProgramData\anaconda3\lib\runpy.py", line 187, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, Error)
File "C:\ProgramData\anaconda3\lib\runpy.py", line 146, in get_module_details
return get_module_details(pkg_main_name, error)
File "C:\ProgramData\anaconda3\lib\runpy.py", line 110, in get_module_details
import(pkg_name)
File "C:\ProgramData\anaconda3\lib\site-packages\bitsandbytes_init.py", line 6, in
python -m bitsandbytes
Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
``
The winamp /add, ansi_red escape, 192.168, oipcam, mp3-svn-repo, claire.sheer.us -- all that stuff is from my environment, but as you can see, it's definitely not in my path. So the error message is too confusing for me to fix here. Or maybe it's in there, and I'm just not noticing it....
Expected behavior
produce an output file without error
Somehow, environment variables not related to this leak into its error messages. So confusing for me.
Hi @ClaireCJS,
Improving this log output and messaging is something we're going to continue to work on in the future. You're right in that this is overly verbose.
As for the issue you're facing specifically, make sure you have a newer version of bitsandbytes installed. Windows is only supported since 0.43.0, and based on this output, you're using an older release.
Hi @ClaireCJS,
Improving this log output and messaging is something we're going to continue to work on in the future. You're right in that this is overly verbose.
As for the issue you're facing specifically, make sure you have a newer version of bitsandbytes installed. Windows is only supported since 0.43.0, and based on this output, you're using an older release.
Thank you very much!
How were you able to tell from that output? Did I just completely miss it?
But also, how are these error messages claiming that things not in my path are? That's a straight-up bug somehow.