direct-pretraining icon indicating copy to clipboard operation
direct-pretraining copied to clipboard

Published 20 hours ago • verified publisher icon wxzs5

Fixed learning rate during pretraining

Open mtchiu2 opened this issue 2 years ago • 0 comments

Hi,

Thank you for your work. I was wondering what is the motivation for using fixed learning rate during pretraining? Is it purely from empirical results or are there any particular reasons?

mtchiu2 avatar May 24 '22 00:05 mtchiu2