Can we directly load a QuaRot-GPTQ quantized model and do lm_eval evaluation?

[Shuai-Xie]

[sashkboos]

Hi,

Thanks for opening this issue.

Actually, you cannot load GPTQ weights into the QuaRot as the approaches are a bit different. In QuaRot, we fuse some randomized Hadamard matrices into the weights before applying quantization which is not the case in the GPTQ. So the quantized QuaRot and GPTQ weights are not the same.

Please let me know if you have any other questions :-)