huggingface
CI: Add gptqmodel to the CI
This issue is to track the TODO from this comment. Once optimum 1.24.0 and transformers 4.49.0 are released, we should enable gptqmodel in the CI (and remove auto-gptq).
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Not stale. New optimum and transformers releases are out, we should be good to proceed.
@BenjaminBossan Affirmative. To be clear, should we clear out all autogptq specific base code or just remove autogptq ci tests and replace with gptqmodel tests only?
@Qubitium @jiqing-feng We forgot a bit about this, I wonder if it's safe now to remove gptq from PEFT completely and fully rely on gptqmodel. What is your opinion?
Also, since https://github.com/ModelCloud/GPTQModel/pull/1703 is merged, I wonder if we can remove auto-awq and also use gptqmodel instead.
@BenjaminBossan @jiqing-feng Yes. I am in agreement with both AutoGPTQ and AutoAWQ deprecation. On the AutoAWQ front, not only have we merged AWQ support but we have sinced added two new AWQ kernels: AwqTorch (100% torch) and AwqTorchFused which is based on the work by @jiqing-feng for optimized cpu ops using Intel avx/xmx/amx instructions (we have since validated the kernel on AMD as well).
There may be Peft compat issues due to latest GPT-QModel refractors but I will work with Peft and @jiqing-feng to fix any issues regarding Peft regardes to kernel selection, training, etc so we have a stable pinned release Peft can count on.
Fantastic, thanks you two. Let us know if we can help with anything.
@BenjaminBossan @jiqing-feng Work has begun on the autogptq and autoawq deprecation.
https://github.com/huggingface/peft/pull/2917 https://github.com/huggingface/transformers/pull/41567
I will ping when the PRs is ready for review and CI are passing.