Deep-Autoencoders-For-Collaborative-Filtering icon indicating copy to clipboard operation
Deep-Autoencoders-For-Collaborative-Filtering copied to clipboard

l2 regularization of weights only

Open Engineero opened this issue 6 years ago • 0 comments

Modified l2_loss term so that it does not count biases, only weights. Typically we do not want to regularize biases as they are:

  1. additive, not multiplicative, and so are far less likely to cause numeric issues if they get too large, and
  2. not necessarily limited by real-world constraints to be near zero.

Basically our data can have biases that are far from zero, so we shouldn't try to keep them close to zero.

Engineero avatar May 29 '18 15:05 Engineero