TinyLlama
TinyLlama copied to clipboard
how to determine reasonable max steps?
Hello, you have a great program! It has been very beneficial! One question I would like to ask is how to determine reasonable max steps based on the amount of data available (e.g. tokens and model parameter count)?or do you have any good ideas in this regard?
Hi you can determine the max steps based on how much tokens you want to train when using cos lr schedule.