CLR
CLR copied to clipboard
Add circular momentum
Based on the new paper of Smith ( https://arxiv.org/abs/1803.09820), I included the momentum variation in your code.
I hope it helps and thank you a lot for your implementation
This is a nice addition to the callback. I have just one question, what if the optimizer used has no momentum property and the user enabled it?
I think self.use_momentum should be a class property where it should take into consideration use_momentum parameter passed by user and check if the self.optimizer has momentum within.