BinaryNet_and_XNORNet icon indicating copy to clipboard operation
BinaryNet_and_XNORNet copied to clipboard

why the neurons are not -1 or 1 after BinaryConv layer?

Open Viscoo opened this issue 6 years ago • 3 comments

I have run the code successful, but when I check the neurons after the binaryconv layer,but it distribute dispersedly,and have no trend to -1 or 1。

Viscoo avatar Apr 08 '18 04:04 Viscoo

The activations are binarised at the input to the next layer.

as for the final layer: my understanding is that whilst the convolution multiplications are binary (xnor operation), the summing is not:

y = W*X + B y = xnor(w1, x1) + xnor(w2, X2) ... + B

yaysummeriscoming avatar Apr 08 '18 18:04 yaysummeriscoming

FYI theres another really good paper that has come out that discusses this a bit further and also has some big improvement on the original BinaryNet implemented here. It's a pretty small extension to BinaryNet, hopefully I'll get some time to do an implementation:

How to Train a Compact Binary Neural Network with High Accuracy? https://www.aaai.org/ocs/index.php/AAAI/AAAI17/paper/download/14619/14454

yaysummeriscoming avatar Apr 08 '18 18:04 yaysummeriscoming

@yaysummeriscoming the results in this paper is impressive. When will you implement it? Thanks.

wjtan99 avatar Apr 26 '18 22:04 wjtan99