h2ogpt
h2ogpt copied to clipboard
h2oai
0% GPU usage WINDOWS 11
Hello, I managed to run the script successfully using the: python generate.py --base_model=h2oai/h2ogpt-gm-oasst1-en-2048-falcon-7b-v3 --langchain_mode=MyData --score_model=None --share=False --prompt_type=human_bot --gradio_offline_level=1 --load_4bit=True
However, today I installed new cuda drivers and something went wrong and after I run the command my GPU is on 0%.
Any ideas what could go wrong?
Thank you!
Hi, I can't tell what could have gone wrong. You can do things like docs/README_GPU.md (https://github.com/h2oai/h2ogpt/blob/162900fba8b03fa30a5a41fd8e11bb8a34131d2f/docs/README_GPU.md) shows, like:
import torch
print(torch.cuda.is_available())
should print True.
That is, it is unlikely a problem specific to h2oGPT.
After trying to modify the cuda version, I started getting this error:
RuntimeError: CUDA Setup failed despite GPU being available. Please run the following command to get more information:
python -m bitsandbytes
Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues
After that I reinstalled my GPU drivers and h2ogpt, then it worked, but my GPU usage is now on 0%.
After running the:
import torch print(torch.cuda.is_available())
by reading the .py file, it actually says CPU, which would print False for the GPU usage. I am confused and have no idea how to fix the issue, I am hoping someone had a similar issue.
Also I forgot to mention that after running:
python -m bitsandbytes I get this error: (h2ogpt) C:\Users\frube\h2ogpt>python -m bitsandbytes
===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
bin C:\Users\frube\miniconda3\envs\h2ogpt\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.dll C:\Users\frube\miniconda3\envs\h2ogpt\lib\site-packages\bitsandbytes\cextension.py:34: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable. warn("The installed version of bitsandbytes was compiled without GPU support. " function 'cadam32bit_grad_fp32' not found CUDA SETUP: Loading binary C:\Users\frube\miniconda3\envs\h2ogpt\lib\site-packages\bitsandbytes\libbitsandbytes_cpu.dll... ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++ BUG REPORT INFORMATION ++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Traceback (most recent call last):
File "C:\Users\frube\miniconda3\envs\h2ogpt\lib\runpy.py", line 196, in run_module_as_main
return run_code(code, main_globals, None,
File "C:\Users\frube\miniconda3\envs\h2ogpt\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\frube\miniconda3\envs\h2ogpt\lib\site-packages\bitsandbytes_main.py", line 95, in
Also when trying to run "pip setup.py install'' I get:
argilla 1.12.1 requires numpy<1.24.0, but you have numpy 1.24.0 which is incompatible. argilla 1.12.1 requires pandas<2.0.0,>=1.0.0, but you have pandas 2.0.2 which is incompatible. After trying to get correct versions for argilla, h2ogpt asks for the previous versions, and it stays like that in a loop. Could that be the issue?
For bitsandbytes, did you use the windows installation direction's version of that package?
For bitsandbytes, did you use the windows installation direction's version of that package?
Yes, I followed the steps for Windows section. It worked fine just yesterday, can't get it to work now no matter what I do.
I revamped and fully tested the docs for windows, please try again: https://github.com/h2oai/h2ogpt/blob/main/docs/README_WINDOWS.md
solution for me was reinstalling cuda drivers hope this helps to future readers
