lmdb read very slow when multi-processing

[bring728]

First of all, thank you for sharing your wonderful code.

Using your code, I'm trying to use pytorch DDP. When num_workers >1 or using DDP, data load may take a very long time. When the lmdb file is caching, the data load takes only about 1ms, but when it is not caching, it sometimes takes tens of seconds. Do you know why?

When reading lmdb in a single process, it reads in about 0.2 seconds even if caching is not enabled.

[C0NGTRI123]

Are you solve that. I run multiprocessing, they are slower than single process