BinaryNet icon indicating copy to clipboard operation
BinaryNet copied to clipboard
verified publisher icon MatthieuCourbariaux

Batchnorm as a single parameter

Open JordyDH opened this issue 6 years ago • 1 comments

Hi, https://www.reddit.com/r/MachineLearning/comments/450a4p/160202830_binarynet_training_deep_neural_networks/czut3dp/ In this reddit post you suggest you can calculate the bn parameter as a threshold. T = mean + β (std + e) / γ My question:

  1. Theano trains inv_std so do i need to do 1.0/inv_std to become std
  2. the "e" value is it the eulers value of : 2.71828

JordyDH avatar Mar 20 '19 14:03 JordyDH

  1. I do not remember.
  2. e is a small number to prevent instability... maybe 10^-6. I do not remember.

MatthieuCourbariaux avatar Mar 17 '24 14:03 MatthieuCourbariaux