Yuxiang Yang
Yuxiang Yang
Close this issue since there are no activities for a long time
Hi @veritas9872, thanks for your attention to MS-AMP. You can set `use_fp32_linear` to toch.nn.Linear if you don't want to use FP8 for this layer. https://github.com/Azure/MS-AMP/blob/9ac98df5371f3d4174d8f103a1932b3a41a4b8a3/msamp/nn/linear.py#L153
Close this issue since there are no activities for a long time
We haven't test MS-AMP with pytorch 22. Currently we only support pytorch1.14 and 2.1. And it is recommended to use our docker image or nvcr.io/nvidia/pytorch:23.10-py3. And we have plan to...
Can you share me the complete steps of reproducing this issue?
When applying FP8 to FSDP, there are 2 problems we need to solve: 1 FSDP requires that all parameters have same dtypes. If we only change some parameters to FP16/FP8,...
Close this issue since there is no activities for a long time.
Hi. Thanks for your attention to our work. Currently we don't have H800 node in hand, so we can't verify it. Have you tried latest MS-AMP docker? Not using `--privileged...
Close this issue since there is no activities for a long time.
Close this issue since there is no activity for a long time.