Comparing performance with PyTorch DataLoader

[vbvg2008]

Hey, great work on the data loader! it's pretty neat and it helped me understand PyTorch's dataloader a lot.

I did some performance comparison between your data loader and PyTorch's loader, and found that the performance difference is pretty big. For example, take a task that takes mnist images and perform a minmax normalization, PyTorch's data loader is around 200 steps/sec on my linux machine whereas tiny dataloader is around 30 steps/sec. I made sure the number of worker is the same between both data loader.

What's your thought on the performance differences and could you point some directions at improving the speed?

Thanks a lot!