very slow in multi-gpu

[YoungDav]

Environmental for requests numpy==1.15.4 graphviz==0.8.4 torch==1.0.0 torchvision==0.2.1 tensorboard==1.13.0 tensorboardX==1.6

when I use 4 gpus(GPU-Util about 4-21%), time will be very very slower than only use 1 gpu(GPU-Util about 30-90%)？

[JiyueWang]

I have the same problem. Have you solved it?

[YoungDav]

I have the same problem. Have you solved it?

no

[mk-minchul]

I faced the same problem and found a solution by using DistributedDataParallel from Nvidia apex.

[luzai]

Hi @mk-minchul , could you share the solution of DistributedDataParallel? It would be quite helpful to increase the batch size and speedup search process by multi-gpu. Thank you very much!