DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

stage_1_and_2.py: do gradient scale only for fp16

Open guoyejun opened this issue 2 years ago • 1 comments

guoyejun avatar Apr 09 '23 10:04 guoyejun

for bf16, the gradient scale is not needed.

guoyejun avatar Apr 09 '23 10:04 guoyejun