Neo LLama

> 全量finetune，ZeRO3，设置output_router_logits=True。训练过程中会突然卡住，GPU利用率突然到100%  请问有解决吗

> hello！ I also met some problems when running the msra2mrc.py?do you know how to deal with the problem of invilid inputs?thank you so much if you could give me...

> ```python > torch.set_default_tensor_type(torch.cuda.HalfTensor) > ``` i have same problem when i train llama3, in modeling_llama.py 1095: ``` causal_mask = torch.triu(causal_mask, diagonal=1) ``` i fix this by : ``` causal_mask...

excuse me ,can you help me convert the raw videos and text to TFRecords,i have downloaded the two datasets  how can i convert them? thank you

> @cooper12121, please share repro steps, such as scripts, full stack trace, and ds_config. **I use transformers' Trainer to train my model, Here is the full error report:** ``` 11.219.19.47:...