text-generation-webui icon indicating copy to clipboard operation
text-generation-webui copied to clipboard

Published 20 hours ago • verified publisher icon oobabooga

Not using CUDA

Open PeterDaGrape opened this issue 2 years ago • 11 comments

I am running windows 10, and ran the install script with no errors, however running the start-webui script it wont find my gpu (RTX 3060 Ti) running conda list reveals no installed cuda package even though I selected Nvidia when installing. Running nvcc also doesnt find anything and ive had a similar issue so I believe its something to do with the path, though I dont understand that. Also checking the torch version reveals 2.0.0+cpu

PeterDaGrape avatar Mar 17 '23 01:03 PeterDaGrape

Try re-downloading the script and starting over, it was updated yesterday

https://github.com/oobabooga/one-click-installers/archive/refs/heads/oobabooga-windows.zip

oobabooga avatar Mar 17 '23 01:03 oobabooga

I downloaded it about 3 hours ago, it should be up to date

PeterDaGrape avatar Mar 17 '23 01:03 PeterDaGrape

Im not familiar with conda, how would I tell it to install cuda and also torch gpu

PeterDaGrape avatar Mar 17 '23 01:03 PeterDaGrape

Also running into this issue. I'm using a rtx 3060 with Windows 11 and just downloaded the script an hour ago.

sbarukes avatar Mar 17 '23 20:03 sbarukes

I have the same issue 3060 ti windows 10, have 3 fresh installs

B33T avatar Mar 17 '23 21:03 B33T

Pretty sure it has something to do with cuda, specifically the version and how conda uses it, I'll probably have another go at it later today

PeterDaGrape avatar Mar 17 '23 22:03 PeterDaGrape

Can you try WSL? There are some 40 issues about CUDA on Windows. WSL should be a smoother experience.

See here the new instructions: https://github.com/oobabooga/text-generation-webui#installation

oobabooga avatar Mar 17 '23 23:03 oobabooga

Ill try it later, currently experimenting with conda and installing cuda, wondering if I can install cuda 11.7 before running the main install script, then trying to do the same with pytorch making sure its compiled for gpu, then if that all fails ill let yk and try wsl

PeterDaGrape avatar Mar 18 '23 00:03 PeterDaGrape

interesting news, from clean install I installed miniconda first, then conda cuda 11.7, and then installed pytorch cuda. Before I would run torch.cuda.is_available() and it would return false, and now it returns true, next step is to download pygmalion and test it out completely (wish me luck)

PeterDaGrape avatar Mar 18 '23 01:03 PeterDaGrape

Can you try WSL? There are some 40 issues about CUDA on Windows. WSL should be a smoother experience.

See here the new instructions: https://github.com/oobabooga/text-generation-webui#installation

I ended up getting this to work after using WSL...kinda. Now having an issue similar to this https://github.com/oobabooga/text-generation-webui/issues/41

I set the RAM limit to 16GB in a config file but it's still getting killed around 80% GPU usage. This is the model I'm attempting to use: https://huggingface.co/chavinlo/alpaca-native

sbarukes avatar Mar 18 '23 02:03 sbarukes

I had this problem too (Windows without WSL) and I think it was because the script installs a different version of cuda than PyTorch 2.0.0 supports. My solution was to use the PyTorch version right before 2.0.0 and it got me past the problem. I ran into some other problems along the way and it ended up taking hours to work through and debug. I wrote some instructions to help.

xNul avatar Mar 19 '23 06:03 xNul

This issue has been closed due to inactivity for 30 days. If you believe it is still relevant, please leave a comment below.

github-actions[bot] avatar Apr 19 '23 16:04 github-actions[bot]