choose warm up iterations problem

[Xinjie-Wei]

Hi, congratulations on your excellent work! I wan to know how many iterations use in warm up in different datasets? Such as , I have 10 thousands images, and batch size is 8, how to choose iterations for warm up in this model? If I use 300 epochs for training and how many epochs use to warm up? Whether there have a fixed ratio?

[sacmehta]

Hi,

A standard practice for warm up is 5-10 epochs. So, you can use this as an initial starting point.

Because our samplers use variable batch sizes, so epoch length is not fixed. Therefore, we recommend you to compute the epoch length using standard batch size and then multiply it with the number of warm-up epochs.

Let's assume that your dataset has 10000 images and batch size is 25 images per GPU. Assuming we have 4 GPUs, so the effective batch size is 100. So, Total number of iterations in 1 epoch would be 10000/ (25 * 4) = 100 iterations. So, if we want to use 5 epochs for warm-up, then 500 iterations (100 iterations per epoch * 5 epochs) would be a good value.

Hope this helps.

[Xinjie-Wei]

Thanks for your detailed reply！