Changlong Yu
Changlong Yu
Thanks for the reply. I would pull a requet later. By the way, I have the following question concerning the preprocess of atomic dataset. Would appreciate your clarification! 1. How...
@seanliu96 Can you help with this question~
can you provide more details like system config, training scripts?
> Is it reproducible? Yes, when I ran the RLOO experiments with [Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct), I met the same issue with Megatron LM 0.4.0 as backend. Will try to increase NCCL timeout...