pytorch-optimizer icon indicating copy to clipboard operation
pytorch-optimizer copied to clipboard

Published 20 hours ago • verified publisher icon jettify

[Feature Request] Adaptive Gradient Clipping

Open kayuksel opened this issue 4 years ago • 2 comments

It would be cool to use AGC from the following paper with any optimizer in this package: https://arxiv.org/abs/2102.06171

kayuksel avatar Feb 14 '21 21:02 kayuksel

Here is an implementation for Adaptive Gradient Clipping: https://github.com/vballoli/nfnets-pytorch/blob/main/nfnets/agc.py

kayuksel avatar Feb 15 '21 19:02 kayuksel

Would you like to create PR?

jettify avatar Feb 22 '21 23:02 jettify