keras-contrib
keras-contrib copied to clipboard
Add learning rate finder
A learning rate finder can easily be implemented as shown in https://arxiv.org/abs/1506.01186 I would make a PR if this functionality seems appropriate to add.
I implemented a simple callback in keras which can be used to find an optimal learning rate. Feel free to use it: https://github.com/LucasAnders1/LearningRateFinder
Hi this link is broken. Could you provide the correct link? @LucasAnders1
@saksham219 https://github.com/LucasAnders1/LearningRateFinder
Here's my version: https://gist.github.com/WittmannF/c55ed82d27248d18799e2be324a79473
Three changes were made:
- Number of iterations is automatically inferred as the number of batches (i.e., it will always run over one epoch)
- Set of learning rates are spaced evenly on a log scale (a geometric progression) using np.geospace
- Automatic stop criteria if current_loss > 10 x lowest_loss