pytorch_DoReFaNet icon indicating copy to clipboard operation
pytorch_DoReFaNet copied to clipboard

Bias is used in conv layers

Open MasLiang opened this issue 4 years ago • 1 comments

Hi~ I noticed that you used bias in conv layers and they are not quantized. Then I read the paper carefully and didn't find anything about the bias. Then I read the code from tensorpack implementation using TensorFlow. Take the Alexnet as the example. Look at this line, you will find that the author first makes the default value of bias of all conv layers to be False, and then at the first conv layer which is not quantized, the auther set the bias as True. But in this Pytorch code, I found in this line, you set the bias to be True. In my point, we need to change it to be False. I hope I express and understand correctly.

MasLiang avatar Dec 22 '20 07:12 MasLiang

Hi @MasLiang Thanks for pointing this out. I think there is another line that needs to be changed. For resnet, this line needs use_bias to be True as it is the first conv2d layer.

The line you mentioned, where use_bias is set to be True, is fine, because when using this implementation of conv2d_Q_fn, use_bias was changed to False, in this line

Best,

MohammedHAlali avatar Nov 30 '21 18:11 MohammedHAlali