nerfstudio icon indicating copy to clipboard operation
nerfstudio copied to clipboard

implemented threadpool mechanics when loading images

Open dozeri83 opened this issue 3 years ago • 1 comments

Hi, I noticed that the loading of the images is not implemented with threads.

Using threadpool with 4 threads improved the loading time for me by a factor of 2-3, when the images are located locally on the machine SSD.

If the images are located on a network drive the the loading time is better by a factor of 3-4.

This is particularly important when there are many images in the dataset and the RAM memory is "limited" ( for me 400 images could not fitt in my 48 GB memory) and the user have to use num_images_to_sample_from and num_times_to_repeat_image fields. In this case the training becomes very IO intensive ( you keep loading images throughout the training).

Thanks

dozeri83 avatar Nov 21 '22 09:11 dozeri83

Feel free to merge when ready.

tancik avatar Nov 22 '22 16:11 tancik