Ranger-Deep-Learning-Optimizer
Ranger-Deep-Learning-Optimizer copied to clipboard
lessw2020
Ranger - a synergistic optimizer using RAdam (Rectified Adam), Gradient Centralization and LookAhead in one codebase
I get the following warning when using ranger with pytorch 1.6.0 ``` /path/Ranger-Deep-Learning-Optimizer/ranger/ranger.py:138: UserWarning: This overload of addcmul_ is deprecated: addcmul_(Number value, Tensor tensor1, Tensor tensor2) Consider using one of...
https://paperswithcode.com/paper/adabelief-optimizer-adapting-stepsizes-by-the
Actually, there is a [pip package](https://pypi.org/project/pytorch-ranger/) but it is based out of a [fork of this repo](https://github.com/mpariente/Ranger-Deep-Learning-Optimizer). I think it would make sense to collate this effort to the main...
Why Ranger is only available on a github but not as an pip package. Wouldn't it be easier for the community to actually use it?
Hi, I had today a relatively long debug session, after I've upgraded my Pytorch Lightning installation, that the training_step wasn't called. It finally turned out, that the problem was that...
Hello, Thank you for your work on these optimizers btw. I was testing a couple out and was performing quite well with the RangerVA originally. Then, when your gradient centralization...
Hi and thanks for the code! I am using your script for my code and while adapting it to PEP8 specs I found a few details that you may want...
TypeError: mean() received an invalid combination of arguments - got (keepdim=bool, dim=tuple, ), but expected one of: * () * (torch.dtype dtype) * (int dim, torch.dtype dtype) didn't match because...
