keras-contrib icon indicating copy to clipboard operation
keras-contrib copied to clipboard

Add learning rate finder

Open aptrishu opened this issue 6 years ago • 4 comments

A learning rate finder can easily be implemented as shown in https://arxiv.org/abs/1506.01186 I would make a PR if this functionality seems appropriate to add.

aptrishu avatar Dec 28 '17 04:12 aptrishu

I implemented a simple callback in keras which can be used to find an optimal learning rate. Feel free to use it: https://github.com/LucasAnders1/LearningRateFinder

LucasAnders1 avatar Feb 27 '18 10:02 LucasAnders1

Hi this link is broken. Could you provide the correct link? @LucasAnders1

saksham219 avatar Apr 13 '18 09:04 saksham219

@saksham219 https://github.com/LucasAnders1/LearningRateFinder

sachinpuranik99 avatar Apr 15 '18 05:04 sachinpuranik99

Here's my version: https://gist.github.com/WittmannF/c55ed82d27248d18799e2be324a79473

Three changes were made:

  • Number of iterations is automatically inferred as the number of batches (i.e., it will always run over one epoch)
  • Set of learning rates are spaced evenly on a log scale (a geometric progression) using np.geospace
  • Automatic stop criteria if current_loss > 10 x lowest_loss

WittmannF avatar Jul 10 '19 22:07 WittmannF