DeepSpeed FP16 fused and unfused grad norm query.

FP16 fused and unfused grad norm query.

Open ShadenSmith opened this issue 4 years ago • 2 comments

Aug 07 '21 00:08 ShadenSmith

This still needs FP32 and ZeRO. And unit tests :-).

Aug 07 '21 00:08 ShadenSmith

Can one of the admins verify this patch?

Jun 09 '22 20:06 rocm-mici

This ended up being added in a later PR

Mar 24 '23 03:03 jeffra