brevitas
brevitas copied to clipboard
Recalculate scale factors without retraining
Hello,
I am trying to fuse Conv & BatchNorm layers after training the model using torch.nn.utils.fusion fuse_conv_bn_eval
. But I got a huge drop in accuracy, which I guess is resulting from the difference in value range after fusion -fused weights are much smaller than unfused ones- hence the scale factor is now totally different and no longer relates to the weights. Is it possible to recalculate the scale factor without retraining?