rethinking-bnn-optimization icon indicating copy to clipboard operation
rethinking-bnn-optimization copied to clipboard

Published 20 hours ago • verified publisher icon plumerai

kernel_initializer="glorot_normal"

Open puhan123 opened this issue 3 years ago • 4 comments
trafficstars

In birealnet.py, I observe that kernel_initializer is set as "glorot_normal" for Conv2d; However, in QuantConv2D, the kernel_initializer is set as "glorot_normal". Kernel_initializer ="glorot_normal" means the kernel weights are all set as 1 ?

Thank you very much.

puhan123 avatar Apr 20 '22 07:04 puhan123

Glorot normal draws samples from a truncated normal distribution centered on 0, so in this case it means binary weights will be randomly set to either -1 or 1.

lgeiger avatar Apr 20 '22 12:04 lgeiger

Thank you very much. I still have one more doubt. Are both Conv2d and QuantConv2D all set to either -1 or 1?

puhan123 avatar Apr 28 '22 02:04 puhan123

Are both Conv2d and QuantConv2D all set to either -1 or 1?

Only weights of QuantConv2D layers will be binarized.

lgeiger avatar Apr 28 '22 11:04 lgeiger

My problem is that I found that the weights of QuantConv2D obtained with initialization method "glorot_normal" is not +1 or -1. "glorot_normal" in Pytorch framework is as below: image

The example is as : image

The value of w is apparently not +1 or -1.
Thank you very much.

puhan123 avatar Apr 29 '22 02:04 puhan123