Your actual batch size is not fixed.

[ArchNew]

The batch size of images you read into the dataloader is fixed. But due to the number of instances vary, the actual batch size of instances also varies. According to my practice, when batch size set to 16, your actual batch size is larger than 32 instances per GPU, which means there are more than 2 instances per image. The advantage of your data-loading strategy is efficiency. Your dataloader reads the image once and crops all instances in that image.

In HRNet, in order to fix the batch size of the instances, they only crop one instance at a time. So if they want to load 32 instances, they need to read the images 32 times. This is extremely inefficient, needs too much CPU resource. In an 8-GPU machine, even you only use 4 GPUs, the task running on the other 4 can be slowed down due to limited CPU resource.

For the sake of efficiency, I prefer your way of loading data. But do you investigate what the varied instance batch size would do to the batch normalization? Personally, I think as long as the actual batch size is big enough, saying no less than 32, the variation is a good thing. The downside is that you have no control over the GPU memory it needs to consume. For the same set batch size, some epochs might have no trouble to run, others might go out of the memory.