grenade icon indicating copy to clipboard operation
grenade copied to clipboard

Remove momentum from Layers

Open HuwCampbell opened this issue 8 years ago • 2 comments

  • Momentum shouldn't be stored in the layers any more. This will free us up to use a broader set of optimisation algorithms. We will however need to provide a class for fast updates and manipulations of learnable parameters.

  • Gradient associated type family shouldn't exist, we'll just return a Network with gradient weights.

  • randomNetwork shouldn't exist. Networks where all layers have a Random instance will also have a Random instance.

HuwCampbell avatar Feb 26 '17 01:02 HuwCampbell

Moreover, I noticed momentum is not serialized, which means saving/resuming training between sessions may be problematic.

claudeha avatar Dec 16 '18 22:12 claudeha

Indeed.

HuwCampbell avatar Dec 16 '18 23:12 HuwCampbell