DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

[BUG] Communication dtype is broken in zero 3

Open tjruwase opened this issue 2 years ago • 1 comments

https://github.com/microsoft/DeepSpeed/blob/0a73e6e6137a91e1b776f725b637f8b37a75f8e7/deepspeed/runtime/zero/stage3.py#L1168

Only works for float32.

tjruwase avatar Feb 03 '23 22:02 tjruwase

@stas00, FYI

tjruwase avatar Feb 03 '23 22:02 tjruwase