[Feature request] Multi-node training

[yjzhong111]

Hi, I have two questions:

1. Can it be used in multi-node training?
2. When will trainer support deepspeed? I have noticed that integrating deepspeed is in to-do list, but do you have the exact time or schedule?

Thank you!

[erogol]

1. It should work for multi-node training.
2. No timeline for deepspeed. why do you need deepspeed for training?

[yjzhong111]

1. It should work for multi-node training.
2. No timeline for deepspeed. why do you need deepspeed for training?

Thanks! But how can I train in multi-node, is there any instructions about it? For deepspeed, I may use some tricks in deepspeed to improve the training performance.