AutoAWQ
AutoAWQ copied to clipboard
Add phi3 support
@casper-hansen Thank you for your invitation.
This PR introduces the support for phi3 for autoawq.
Due to the fact that the phi3 hasn't been released to transformer package, I conducted experiments on the development branch, a.k.a 4.41.0.dev0
I conducted experiments on RTX4090 and evaluated the perpelexity of the quantized phi3 (microsoft/Phi-3-mini-128k-instruct).