gpt-neo-fine-tuning-example icon indicating copy to clipboard operation
gpt-neo-fine-tuning-example copied to clipboard

Published 20 hours ago • verified publisher icon dredwardhyde

Deepspeed stuck

Open SamsTheGreatest opened this issue 4 years ago • 0 comments

When replicating the code Deepspeed gets stuck with [2021-06-29 14:29:44,757] [INFO] [utils.py:13:_initialize_parameter_parallel_groups] data_parallel_size: 1, parameter_parallel_size: 1

Any ideas on how to fix this?

SamsTheGreatest avatar Jun 29 '21 15:06 SamsTheGreatest