ScaleLLM icon indicating copy to clipboard operation
ScaleLLM copied to clipboard

Published 20 hours ago • verified publisher icon vectorch-ai

Is there any plans to support Int8 weight quant ?

Open sitabulaixizawaluduo opened this issue 7 months ago • 2 comments

The int8 quantization is basically lossless at the moment, and although awq is good in terms of accuracy and performance, int8 is a better choice in some scenarios where accuracy is more important. So I'd like to ask if there are plans to support this feature

sitabulaixizawaluduo avatar Jul 18 '24 12:07 sitabulaixizawaluduo