training memory

[rongleiji]

Hi Xintao, How much memory did you need to train? I tried training by using the script provided in your repo and reserved 400G for training. WIth 8 gpus it exceeded memory limit at epoch 0 iter 4200 and with 2 gpus it exceeded memory limit at epoch 0 iter 11300. So it Is hard to replicate without proper settings.