starcoder icon indicating copy to clipboard operation
starcoder copied to clipboard

Have a question about learning rate decay?

Open SerenNoble opened this issue 2 years ago • 1 comments
trafficstars

Why is it that I set the learning rate decay type in config.yaml, but the learning rate does not change in the log file output when the model is trained?

SerenNoble avatar Aug 21 '23 08:08 SerenNoble

Hi, Can you please elaborate? You should be able to set the learning rate scheduler type by using --lr_scheduler_type. By default it is set to cosine.

ArmelRandy avatar Aug 22 '23 12:08 ArmelRandy