EcoDepth CUDA OUT OF MEMORY during training

          Sorry for the late reply. Thanks for the help. It fixed the inference part. I was able to run inference in the size of 1000*360 as well.  Can you tell me if the same is possible to train the model with the dataset? During the training as well the code gives me CUDA OUT OF MEMORY ERROR!!!

Originally posted by @PreyumKr in https://github.com/Aradhye2002/EcoDepth/issues/18#issuecomment-2395714533

Oct 08 '24 08:10 PreyumKr

Hi @PreyumKr,

Might I ask what per device batch size you used for training? Also, how much memory does (do) your GPU(s) have? As per our experiments, for the smallest per device batch size of 1, 15 GB of GPU memory should be sufficient.

Oct 31 '24 04:10 Aradhye2002

I did change batch size in the training script from 4 to 1 but it didn't help. The GPU is 16gb but still the memory runs out. I tried training on the nyu v2 dataset.

Oct 31 '24 07:10 PreyumKr