starcoder
starcoder copied to clipboard
Have a question about learning rate decay?
trafficstars
Why is it that I set the learning rate decay type in config.yaml, but the learning rate does not change in the log file output when the model is trained?
Hi, Can you please elaborate? You should be able to set the learning rate scheduler type by using --lr_scheduler_type. By default it is set to cosine.