text-generation-webui icon indicating copy to clipboard operation
text-generation-webui copied to clipboard

Compatable with newer gptq

Open catalpaaa opened this issue 1 year ago • 2 comments

yea they are changing up shits again

catalpaaa avatar Apr 20 '23 15:04 catalpaaa

Thanks. I don't have a triton installation to test, so I'll tag @sgsdxzy and @EyeDeck for review.

oobabooga avatar Apr 20 '23 16:04 oobabooga

Thanks. I don't have a triton installation to test, so I'll tag @sgsdxzy and @EyeDeck for review.

sure, they just moved modelutils into utils/ i just made it so it looks for that too

catalpaaa avatar Apr 20 '23 17:04 catalpaaa

Works for me.

Without this patch, the latest GPTQ-for-LLaMA crashes with:

Traceback (most recent call last):
  File "/home/anon/text-generation-webui/server.py", line 912, in <module>
    shared.model, shared.tokenizer = load_model(shared.model_name)
  File "/home/anon/text-generation-webui/modules/models.py", line 125, in load_model
    from modules.GPTQ_loader import load_quantized
  File "/home/anon/text-generation-webui/modules/GPTQ_loader.py", line 15, in <module>
    from modelutils import find_layers
ModuleNotFoundError: No module named 'modelutils'

Maybe some day we'll stop needing to fight with random breaking changes every three days :')

EyeDeck avatar Apr 21 '23 08:04 EyeDeck

I merged #1451 which had the simpler import syntax

oobabooga avatar Apr 21 '23 15:04 oobabooga

I did not see this pull mybad.

USBhost avatar Apr 21 '23 15:04 USBhost