BitNet icon indicating copy to clipboard operation
BitNet copied to clipboard

Fix: Weight quantization sign should be the last operation

Open jmbrito01 opened this issue 8 months ago • 1 comments

Description: This PR fixes a bug in the weight_quant() function, where the weights do not follow the paper orientation to when to sign the weights. Basically the sign operation should be the last thing to be done but it's being done before the multiplication of the scale(see this for more informations)

For discussion: The weights and activation are still being done at full precision on BitLinear, it's probably best to convert it to int8 like the paper proposes to provide better speed.

Issue: #52

jmbrito01 avatar Jun 27 '24 20:06 jmbrito01