open_lm icon indicating copy to clipboard operation
open_lm copied to clipboard

How to pretrain on DCLM-BASELINE

Open mathfinder opened this issue 5 months ago • 0 comments

Thank you for your excellent work. If I want to use DCLM-BASELINE for pretraining and conduct a rigorous comparison with the DCLM-BASELINE 7B model, what hyper-parameters should I use? Could you provide the corresponding script? Thank you.

mathfinder avatar Aug 30 '24 16:08 mathfinder