text-generation-webui
text-generation-webui copied to clipboard
Compatable with newer gptq
yea they are changing up shits again
Thanks. I don't have a triton installation to test, so I'll tag @sgsdxzy and @EyeDeck for review.
Thanks. I don't have a triton installation to test, so I'll tag @sgsdxzy and @EyeDeck for review.
sure, they just moved modelutils into utils/ i just made it so it looks for that too
Works for me.
Without this patch, the latest GPTQ-for-LLaMA crashes with:
Traceback (most recent call last):
File "/home/anon/text-generation-webui/server.py", line 912, in <module>
shared.model, shared.tokenizer = load_model(shared.model_name)
File "/home/anon/text-generation-webui/modules/models.py", line 125, in load_model
from modules.GPTQ_loader import load_quantized
File "/home/anon/text-generation-webui/modules/GPTQ_loader.py", line 15, in <module>
from modelutils import find_layers
ModuleNotFoundError: No module named 'modelutils'
Maybe some day we'll stop needing to fight with random breaking changes every three days :')
I merged #1451 which had the simpler import syntax
I did not see this pull mybad.