puzzle on training speed

[ChristianoPenaldo]

I train the model on a Tesla M40 GPU. However, I could only train 40 steps within 1 minute. I admit that Titan Xp is faster than M40, so is multi-GPU training feasible?

[jiangsutx]

Yes. I think multi-gpu is possible. We use only common modules, which should be easy to put on multi-gpu. But the current codes need to be modified to use multi-gpu.

[chenxinwhut]

How to modify the code to meet multi-gpu training