Sam Vance

I am having the same issue, except in the debug, GOT UDP! shows up in lists of 10 or more

I was able to get the model to run by first converting the weights to deepspeed checkpoints, and then loading the model from those checkpoints. I set deepspeed strategy as...

@HeorhiiS It was a full 7B model. Note that it trained slower than the normal model.