grenade
grenade copied to clipboard
Remove momentum from Layers
-
Momentum shouldn't be stored in the layers any more. This will free us up to use a broader set of optimisation algorithms. We will however need to provide a class for fast updates and manipulations of learnable parameters.
-
Gradientassociated type family shouldn't exist, we'll just return a Network with gradient weights. -
randomNetworkshouldn't exist. Networks where all layers have aRandominstance will also have aRandominstance.
Moreover, I noticed momentum is not serialized, which means saving/resuming training between sessions may be problematic.
Indeed.