Qwen2.5 icon indicating copy to clipboard operation
Qwen2.5 copied to clipboard

Published 20 hours ago • verified publisher icon QwenLM

Qwen1.5-7B-Chat AWQ量化的MMLU评测效果相比Qwen1.5-7B-Chat-GPTQ-Int4和Qwen1.5-7B-Chat相差特别大

Open luchangli03 opened this issue 9 months ago • 3 comments

我评测了Qwen1.5-7B-Chat和两个量化模型的MMLU效果，发现AWQ的分数特别低，比直接naive 4bit还差。这是什么情况呢？ 浮点模型分数0.60，而GPTQ版本0.59而AWQ版本只有0.45，naive的版本都有0.589 GPTQ和AWQ量化模型: https://huggingface.co/Qwen/Qwen1.5-7B-Chat-AWQ https://huggingface.co/Qwen/Qwen1.5-7B-Chat-GPTQ-Int4

luchangli03 avatar Apr 26 '24 07:04 luchangli03