CUDA out of memory

[hhboy98]

Thank you for your sharing. Why do I break out CUDA out of memory every time in a fixed training stage

[vcjmhg]

@hhboy98 , I also meet this problem. Have you solved it?

[YuvalNirkin]

Try to reduce the batch size.

[vcjmhg]

ok, thank u!

[leo-hao]

Hi，a new problem has appeared again, even though I set the batch_size to 1, it still out of memory.

[YuvalNirkin]

It seems like your GPU does not have enough memory (10GB) for running the training even with batch_size set to 1. In the configuration you have attempted to run the training resolution is 1024x512 this is quite a large resolution, you can try reducing it to 512x256 and you should still get reasonable results. Alternatively you try training a smaller model, for example using the the B0 backbone instead of the B1 backbone.

[leo-hao]

Dear doctor Nirkin, I know where I am wrong, I need to modify the batch-size in /config/train/cityscapes_efficientnet_b1_hyperseg-m.py, instead of writing --batch-size 1 in the terminal.
With the help of your tips, I finally started my training successfully. Sincerely thank you for your patience and help. Wish you a happy life and smooth work.