Long training time

[SChoi97]

Dear Sapient Inc Team.

First of all, thank you very much for the incredible work.

I have a question about the long training time. According to the configs, the models are trained for 100k epochs with a batch size of ~700 suggesting more than 100k training iterations for a 1000 sample dataset. Did you find that the reasoning capabilities and the capacity for inference time scaling emerged after long training times or do early checkpoints also display wthis behaviour? Does the model gradually learn to use M segments over long training iterations?

Thank you very much, once again.