JackieWu issues

Repositories
Issues
Comments

Results 12 issues of


                                            JackieWu

[Feature] Auto scaling factor tuning for FP8 collective communication

**Description** Support for auto scaling factor tuning #41 Related Example: https://github.com/Azure/MS-AMP-Examples/pull/21 Performance (model: GPT-345M, https://github.com/Azure/MS-AMP-Examples/blob/main/gpt3/pretrain_345m_megatron.sh): - msamp w/o auto scaling validation loss at iteration 5000 | lm loss value: 3.531525E+00...

[iRPE] fix rpe_ops import

1. Fix the issue #234 2. Support for PyTorch 2.x