Haebin Shin

Thanks for great work. I want to use it with LLaMA2 or LLaMA3, but I'm wondering if I need to train it again or if there's a way to use...

Hi, I'm experiencing performance degradation when using multi-node training with `pretrain.py` I followed the [continual pretraining tutorial](https://lightning.ai/lightning-ai/studios/continued-pretraining-with-tinyllama-1-1b) using TinyLlama on the OpenWebMath 14B dataset. I'm working with a bare-bones multi-node...