need help have a 3090 wont work

[dangelo352]

(minigpt4) C:\MINIGPT\MiniGPT-4>python demo.py --cfg-path eval_configs/minigpt4_eval.yaml --gpu-id 0 Initializing Chat Loading VIT Loading VIT Done Loading Q-Former Loading Q-Former Done Loading LLAMA

===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues

C:\Users\Dangelo\anaconda3\envs\minigpt4\lib\site-packages\bitsandbytes\cuda_setup\main.py:141: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('C:/Users/Dangelo/anaconda3/envs/minigpt4/bin')} warn(msg) C:\Users\Dangelo\anaconda3\envs\minigpt4\lib\site-packages\bitsandbytes\cuda_setup\main.py:141: UserWarning: C:\Users\Dangelo\anaconda3\envs\minigpt4 did not contain cudart64_110.dll as expected! Searching further paths... warn(msg) C:\Users\Dangelo\anaconda3\envs\minigpt4\lib\site-packages\bitsandbytes\cuda_setup\main.py:141: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('%LD_LIBRARY_PATH%')} warn(msg) C:\Users\Dangelo\anaconda3\envs\minigpt4\lib\site-packages\bitsandbytes\cuda_setup\main.py:141: UserWarning: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\libnvvp;%LD_LIBRARY_PATH%;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\libnvvp;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\libnvvp did not contain cudart64_110.dll as expected! Searching further paths... warn(msg) C:\Users\Dangelo\anaconda3\envs\minigpt4\lib\site-packages\bitsandbytes\cuda_setup\main.py:141: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {WindowsPath('C:/Users/Dangelo/anaconda3/envs/myenv/Library/mingw-w64/bin'), WindowsPath('C:/Users/Dangelo/anaconda3/envs/myenv/Library/usr/bin'), WindowsPath('C:/Users/Dangelo/anaconda3/envs/myenv/bin'), WindowsPath('C:/Program Files/node'), WindowsPath('C:/Users/Dangelo/anaconda3/envs/myenv/Library/bin'), WindowsPath('C:/Users/Dangelo/anaconda3/envs/minigpt4/Lib/site-packages/cv2/../../x64/vc14/bin'), WindowsPath('C:/Users/Dangelo/anaconda3/envs/myenv/Scripts'), WindowsPath('C:/Users/Dangelo/anaconda3/envs/minigpt4/Library/usr/bin'), WindowsPath('C:/Users/Dangelo/anaconda3/envs/minigpt4/bin'), WindowsPath('C:/Users/Dangelo/anaconda3/envs/myenv'), WindowsPath('C:/Users/Dangelo/anaconda3/envs/minigpt4/Library/mingw-w64/bin')} warn(msg) CUDA SETUP: CUDA runtime path found: C:\Users\Dangelo\anaconda3\envs\minigpt4\Library\bin\cudart64_110.dll CUDA SETUP: Highest compute capability among GPUs detected: 8.6 CUDA SETUP: Detected CUDA version 113 CUDA SETUP: Required library version not found: libbitsandbytes_cuda113.dll. Maybe you need to compile it from source? CUDA SETUP: Defaulting to libbitsandbytes_cpu.dll... C:\Users\Dangelo\anaconda3\envs\minigpt4\lib\site-packages\bitsandbytes\cextension.py:31: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable. warn("The installed version of bitsandbytes was compiled without GPU support. " Traceback (most recent call last): File "C:\MINIGPT\MiniGPT-4\demo.py", line 59, in model = model_cls.from_config(model_config).to('cuda:0') File "C:\MINIGPT\MiniGPT-4\minigpt4\models\mini_gpt4.py", line 241, in from_config model = cls( File "C:\MINIGPT\MiniGPT-4\minigpt4\models\mini_gpt4.py", line 89, in init self.llama_model = LlamaForCausalLM.from_pretrained( File "C:\Users\Dangelo\anaconda3\envs\minigpt4\lib\site-packages\transformers\modeling_utils.py", line 2740, in from_pretrained raise ValueError( ValueError: Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit the quantized model. If you want to dispatch the model on the CPU or the disk while keeping these modules in 32-bit, you need to set load_in_8bit_fp32_cpu_offload=True and pass a custom device_map to from_pretrained. Check https://huggingface.co/docs/transformers/main/en/main_classes/quantization#offload-between-cpu-and-gpu for more details.

[FrancescoSaverioZuppichini]

You need to use cuda 11.8, use docker, it's easier - you can look for bitsandbytes related issues on the bitsandbytes repo

[FrancescoSaverioZuppichini]

https://github.com/TimDettmers/bitsandbytes/issues/112

[kelheor]

Fix your bitsadnbytes package (C:\Users\yourusername\anaconda3\envs\minigpt4\Lib\site-packages\bitsandbytes): https://github.com/DeXtmL/bitsandbytes-win-prebuilt Download dlls and read this: https://github.com/Vision-CAIR/MiniGPT-4/issues/28#issuecomment-1514181638 Then execute in case if you have error that torch is not compiled with CUDA support: pip uninstall torch pip cache purge pip install torch -f https://download.pytorch.org/whl/torch_stable.html pip uninstall torchaudio pip install torchaudio

[dangelo352]

You need to use cuda 11.8, use docker, it's easier - you can look for bitsandbytes related issues on the bitsandbytes repo

Do i need to be using WSL? Ive been using the standard command prompt inside a Conda instance.