MatejUlcar

Megatron: NameError: name 'ensure_divisibility' is not defined

1

**Describe the bug** Trying to run `megatron_gpt_pretraining.py`. During the init, the NameError occurs, Traceback below: ``` Traceback (most recent call last): File "/NeMo/examples/nlp/language_modeling/megatron_gpt_pretraining.py", line 92, in main() File "/NeMo/nemo/core/config/hydra_runner.py", line...

Deepspeed zero optimizer, error converting model checkpoints

2

**Describe the bug** Unable to convert a custom gpt-neox model (with zero stage 3) checkpoints using zero_to_fp32.py script. **To Reproduce** Train a model with zero stage 3, pp=0, mp=1 (haven't...

deepspeed_to_megatron several issues

3

1. A recent commit removed `tools/convert_checkpoint/deepspeed_checkpoint.py` but there is still an attempt to import it in `tools/convert_checkpoint/deepspeed_to_megatron.py`. The other scripts in the folder appear to be ok. I guess the...