GPTQ-for-LLaMa
GPTQ-for-LLaMa copied to clipboard
qwopqwop200
4 bits quantization of LLaMa using GPTQ
I don't know if the error I'm getting is related to GPTQ-for-LLaMa, but is worth a try! This is the scenario: Linux Mint based ubuntu focal 2x AMD RX5700XT I'm...
I'm on Windows 10, using conda, and following instructions from here https://github.com/oobabooga/text-generation-webui/wiki/LLaMA-model Running `python setup_cuda.py install` from the `cuda` branch give this error `TypeError: expected string or bytes-like object` I've...
trying to run text-generation-webui followed mkdir repositories cd repositories git clone https://github.com/oobabooga/GPTQ-for-LLaMa.git -b cuda cd GPTQ-for-LLaMa python setup_cuda.py install then using (after git cloning and git lfs pull) python /mnt/distvol/text-generation-webui/server.py...
openbmb/minicpm is one of best of small model in China，I want to pull request to support minicpm，Can I please do this?
After quantization of LLaMA2-7b, I notice that total parameters of the quantized model is around 1.1B while the original dense model has around 6.7B parameters. It seems that the code...
