smoothquant icon indicating copy to clipboard operation
smoothquant copied to clipboard

How to quantize the out_proj and fc2 module in OPT model family

Open yanchenmochen opened this issue 6 months ago • 0 comments

I am going to make an experiments to quantize opt model family, want to using the smoothquant algorithm, but because there is an activation function between the fc1 and fc2, how to handle fc2. Also, why the code in the repository doesnot quantize the out_proj module?

yanchenmochen avatar Jul 30 '24 03:07 yanchenmochen