sosofun

need a distributed training example

3

Thank you for your innovative work， can you provide a distributed training example? then can quickly reproduct and verify thesis work。

Support for latest Megatron-LM and transformer-engine 1.0 +

2

Thank you for such a great and exciting project ! **What would you like to be added**: Support for latest Megatron-LM and transformer-engine 1.0 + **Why is this needed**: latest...