DeepPavlov icon indicating copy to clipboard operation
DeepPavlov copied to clipboard

Published 20 hours ago • verified publisher icon deeppavlov

Create a Trainer class for learning rate scheduling

Open yoptar opened this issue 5 years ago • 2 comments

For now learning rate scheduling is implemented using the LRScheduledModel class and its inheritors. It would be great to move this logic to a trainer class inherited from the NNTrainer class.

yoptar avatar Apr 15 '19 11:04 yoptar

LRScheduledModel performs

  • learning rate and momentum updating
  • fitting of learning rate on a small subset of data (before main phase of training), see LRScheduledModel.fit for details.

vikmary avatar Apr 15 '19 11:04 vikmary

FYI https://www.tensorflow.org/alpha/guide/keras/custom_callback#learning_rate_scheduling

nikolay-bushkov avatar Apr 15 '19 13:04 nikolay-bushkov