autoclip

autoclip copied to clipboard

Has there been any research on how this strategy interacts with a learning rate schedule? Especially for something extreme like the one-cycle policy (super convergence). It seems like the history...

Permafacture

Gradient_transformers was removed from Tensorflow rendering the algorithm unusable. Is there any alternative ?

DanielWicz

First off, thank you for the wonderful paper and technique. It solves some needs I have been having with a couple of NLP projects, amongst others. I'm making this pull...

HesitantlyHuman

example in PyTorch

2

Hello How are you? Thanks for contributing to this project. Could u provide an example of how to use in PyTorch? Thanks

rose-jinyang

buffer length for history

1

Hello How are you? Thanks for contributing to this project. I made my own AutoClipper class based on your code.  Please check if there is any problem. Here I...

rose-jinyang

Unexpected keyword argument passed to optimizer: gradient_transformers

1

``` learning_rate=0.001, gradient_transformers=[AutoClipper(10)] File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/adam.py", line 115, in __init__ super(Adam, self).__init__(name, **kwargs) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py", line 303, in __init__ "passed to optimizer: " + str(k)) ``` TypeError: Unexpected keyword argument passed...

saurabhkumar8112