InternLM-XComposer icon indicating copy to clipboard operation
InternLM-XComposer copied to clipboard

[Implementation details] layer wise learning rate decay.

Open John-Ge opened this issue 1 year ago • 0 comments

In the paper, you mention learning rate decay is important. I would like to know how could I set layer decay in the code. Do you have plans to release the code of learning rate decay.

John-Ge avatar Feb 08 '24 02:02 John-Ge