Change to multi card training

[xzr52]

Dear author, this is a great job, but I encountered a problem. When I wanted to change from single card training to multi card training, my GPUs did not train together. Do you know the reason or can you provide a solution? Thank you very much for your answer