keras-contrib
keras-contrib copied to clipboard
Add AdaMod optimizer
I'd like to add the AdaMod optimizer to the keras-contrib optimizers. Paper reference: https://arxiv.org/abs/1910.12249 I modified the Adam optimizer code from the main keras repo and added the exponential averaging of past learning rates via the beta_3 coefficient and clamping of learning rates as described by the paper. Here is my current branch: https://github.com/mffigueroa/keras-contrib/commits/user/mffigueroa/adamod