the efficient of duo_scaling

[Skyseaee]

I have been experimenting with duo_scaling using the formula scales = (act_scales.pow(alpha) / w_scales.pow(1 - alpha)).clamp(min=1e-4).to(device).to(dtype) in auto AWQ. However, I observed a 1% decrease in accuracy when tested on the TextVQA dataset. I am looking for references or details regarding the use of scale / weight as opposed to using just scale to better understand this decline. Thanks for answering.