FusedAdam Hangs during Initialization

[lipiji]

I am testing the accelerate training code and I modify the optimizer of: https://github.com/huggingface/accelerate/blob/main/examples/nlp_example.py#L141 to the FusedAdam (from deepspeed.ops.adam import FusedAdam), then after "python ./nlp_example.py" I find that the progress hands on the line of "optimizer = FusedAdam(params=model.parameters(), lr=lr)".

What are the possible reasons? https://github.com/huggingface/accelerate/tree/main/examples

[jomayeri]

Hi @lipiji I was unable to reproduce the bug. Could you provide information about the environment setup you are using?

[jomayeri]

Closing now. Feel free to reopen if needed.