Necofish

Poor Performance of W4A4 on MMLU task

1

I evaluated the MMLU performance of LLaMA 3.1-8B and LLaMA 2-7B models under ABQ W4A4 quantization. The results are as follows, showing a significant drop in MMLU performance. | Model...