Megatron-LM icon indicating copy to clipboard operation
Megatron-LM copied to clipboard

Reload optimizer params when loading bf16 ckpt

Open RookieHong opened this issue 1 year ago • 0 comments

This PR is to fix this issue, when using bf16 and attempting to load pretrained model weights to continue training, the weights in the optimizer will not be reset to the pretrained model weights.

RookieHong avatar Jan 03 '24 03:01 RookieHong