GPTQ models do not work anymore in the latest version of text generation webui

[CRCODE22]

Describe the bug

GPTQ models will not load anymore you only get the press any key to continue to continue and the command prompt closes.

INFO:Gradio HTTP request redirected to localhost :) bin D:\oobabooga_windows\installer_files\env\lib\site-packages\bitsandbytes\libbitsandbytes_cuda117.dll INFO:Loading reeducator_vicuna-13b-free... INFO:Found the following quantized model: models\reeducator_vicuna-13b-free\vicuna-13b-free-v230502-4bit-128g.safetensors

Done! Press any key to continue . . .

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

Just follow the steps accordingly on the github page to install text generation web ui with the one click installer on the page. Then try to load any 13B GPTQ model.

Also when running the update_windows.bat file you will get this error: FileNotFoundError: [WinError 2] The system cannot find the file specified: 'setup_cuda.py' -> 'setup.py'

Screenshot

No response

Logs

start_windows.bat
INFO:Gradio HTTP request redirected to localhost :)
bin D:\oobabooga_windows\installer_files\env\lib\site-packages\bitsandbytes\libbitsandbytes_cuda117.dll
INFO:Loading reeducator_vicuna-13b-free...
INFO:Found the following quantized model: models\reeducator_vicuna-13b-free\vicuna-13b-free-v230502-4bit-128g.safetensors

Done!
Press any key to continue . . .

update_windows.bat
  File "D:\oobabooga_windows\webui.py", line 175, in <module>
    update_dependencies()
  File "D:\oobabooga_windows\webui.py", line 133, in update_dependencies
    os.rename("setup_cuda.py", "setup.py")
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'setup_cuda.py' -> 'setup.py'

System Info

Windows 11
Nvidia Geforce 3060 12 GB Gaming OC 2.0
Processor: AMD Ryzen 7 3700X 8-Core Processor (16 CPUs), ~3.6GHz
Memory: 57344MB RAM
Available OS Memory: 57250MB RAM

[Syncriix]

Same issue on latest, win10 stock.

[Manimap]

I was gonna post about an adjacent problem, since it works for me I think, I used the one click install and just updated, and got this message when updating :

Traceback (most recent call last): File "D:\text-generation-webui\webui.py", line 178, in update_dependencies() File "D:\text-generation-webui\webui.py", line 133, in update_dependencies os.rename("setup_cuda.py", "setup.py") FileNotFoundError: [WinError 2] The system cannot find the file specified: 'setup_cuda.py' -> 'setup.py'

So my guess is that you should update your dependencies ? Or maybe try to rename manually the file "setup_cuda.py" to "setup.py" in text-generation-webui\repositories\GPTQ-for-LLaMa

[Tom-Neverwinter]

https://github.com/jllllll/one-click-installers

made pull request and updated all oobabooga web ui or download and replace files. https://github.com/oobabooga/text-generation-webui/commits/main

solved with updated installer and cuda installed 11.8[tesla m40] and 12.1 [modern cards]

[LiveRock]

Do you have a tutorial for a newbie? Facing the same

[Tom-Neverwinter]

Do you have a tutorial for a newbie? Facing the same

https://www.youtube.com/watch?v=FOyqcETVUCs

[thezveroboy]

i have the same problem on my 12Gb card with the last updates of oobabooga. what's going on? the new version wants to eat all the memory? previous versions had work fine.

[github-actions[bot]]

This issue has been closed due to inactivity for 6 weeks. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.