aphrodite-engine icon indicating copy to clipboard operation
aphrodite-engine copied to clipboard

[Feature]: Support hqq quantize method.

Open Minami-su opened this issue 1 year ago • 0 comments

🚀 The feature, motivation and pitch

https://mobiusml.github.io/hqq_blog/

HQQ is a fast and accurate model quantizer that skips the need for calibration data. It's super simple to implement (just a few lines of code for the optimizer). It can crunch through quantizing the Llama2-70B model in only 4 minutes! 🚀

Alternatives

No response

Additional context

No response

Minami-su avatar Apr 17 '24 14:04 Minami-su