bagua icon indicating copy to clipboard operation
bagua copied to clipboard

Published 20 hours ago • verified publisher icon BaguaSys

support amp with fused optimizer

Open wangraying opened this issue 3 years ago • 2 comments

torch native amp + apex amp

wangraying avatar Nov 16 '21 10:11 wangraying

Hi! Any updates on this issue? Is it possible to use fuse_optimizer with torch.cuda.amp.GradScaler?

windj007 avatar Feb 26 '22 21:02 windj007

Not yet. We may solve this issue in Q2 this year, hopefully. There is a previous draft related to this issue https://github.com/BaguaSys/bagua/compare/master...fuse-opt-fp16.

wangraying avatar Feb 27 '22 12:02 wangraying