Extremely slow data load when training on ImageNet

[gasvn]

trafficstars

Thanks for your excellent work. I found that the data load speed is very slow when training the shuffleNetV2-0.5X. The GPU is basically not working. I use one SSD and 10 workers to load data. Normally the data load for a batch of 256 requires less than 0.1s, but it becomes more than 10s now. It seems that GPU is much more fast than the data loading process. I wonder if you have solution for this problem? Thanks.

[Hou9612]

Thanks for your excellent work. I found that the data load speed is very slow when training the shuffleNetV2-0.5X. The GPU is basically not working. I use one SSD and 10 workers to load data. Normally the data load for a batch of 256 requires less than 0.1s, but it becomes more than 10s now. It seems that GPU is much more fast than the data loading process. I wonder if you have solution for this problem? Thanks.

Hi, I meat the same problem. It seems that training one epoch needs more than 12 hours and I use two 1080Ti . Have you solve this problem yet?