Megatron-LM icon indicating copy to clipboard operation
Megatron-LM copied to clipboard

Published 20 hours ago • verified publisher icon NVIDIA

fix: count_zeros protection in chained optimizer

Open clumsy opened this issue 6 months ago • 2 comments

Make ChainedOptimizer honor log_num_zeros_in_grad to keep the behavior consistent in case we silently end up using it, e.g. when using EP>1.

clumsy avatar May 08 '25 14:05 clumsy