AutoAWQ icon indicating copy to clipboard operation
AutoAWQ copied to clipboard

Add phi3 support

Open pprp opened this issue 1 year ago • 0 comments

@casper-hansen Thank you for your invitation.

This PR introduces the support for phi3 for autoawq.

Due to the fact that the phi3 hasn't been released to transformer package, I conducted experiments on the development branch, a.k.a 4.41.0.dev0

I conducted experiments on RTX4090 and evaluated the perpelexity of the quantized phi3 (microsoft/Phi-3-mini-128k-instruct).

image

pprp avatar May 13 '24 02:05 pprp