Activation for BinaryNet

[ruizhoud]

Hello,

I noticed that torch.nn.Hardtanh is used for activation functions in BinaryNet. This is meant to make the model trainable, as introduced in the BNN paper. However, in the inference phase (the validate() function in main_binary.py), shoudn't the activation function be changed to sign function so that the intermediate results are binary?

Thanks!

[roeemz]

Check out binarized_modules: https://github.com/itayhubara/BinaryNet.pytorch/blob/master/models/binarized_modules.py#L76 and https://github.com/itayhubara/BinaryNet.pytorch/blob/master/models/binarized_modules.py#L95

You will see that the input to each layer is Binarized anyway so as long as you use BinarizeConv2d or BinarizeLinear you are good.