borrero-c

Also having this problem with `train_finetune_accelerate.py`. I haven't dug too deep but the `accelerator.backward()` calls seemed to be taking a very long time, specifically this code block https://github.com/yl4579/StyleTTS2/blob/5cedc71c333f8d8b8551ca59378bdcc7af4c9529/train_finetune_accelerate.py#L449-L464

Also showing low GPU utilization and high single core CPU utilization  It also seems like the issue goes away after the first epoch is finished, my...

Looked into it some more, my steps are taking 40-20 seconds long and the `.backwards()` call is taking 20-10 seconds respectively. When the training starts to pick up after that...