text-generation-webui icon indicating copy to clipboard operation
text-generation-webui copied to clipboard
verified publisher icon oobabooga

GPTQ-GPTJ

Open ewof opened this issue 3 years ago • 3 comments

https://github.com/AlpinDale/gptq-gptj/commits/main INT4 AND SHIET FOR PYG

ewof avatar Mar 19 '23 16:03 ewof

Huge

oobabooga avatar Mar 19 '23 16:03 oobabooga

So now there is NEO-X and GPTJ and Regular GPTQ (opt,llama,bloom).. all separate repo with separate kernels. And I think the kernels may not work across versions and all require install. Haven't tried if they can co-exist.

When I renamed the bad pull of GPTQ this morning it wasn't enough to rename the folders, I had to rebuild and install the kernel again.

Ph0rk0z avatar Mar 19 '23 19:03 Ph0rk0z

Everyone should just PR to https://github.com/qwopqwop200/GPTQ-for-LLaMa imo.

oobabooga avatar Mar 19 '23 22:03 oobabooga