Megatron-DeepSpeed
Megatron-DeepSpeed copied to clipboard
Tweaks for lm-eval-harness
- Only check for
position_embedding_type
if the field exists for the checkpoint-loaded args. - Only load optimizer/lr scheduler states if user provide optimizer and lr scheduler.