RuntimeError: CUDA out of memory. Tried to allocate 900.00 MiB (GPU 0; 10.92 GiB total capacity; 7.83 GiB already allocated; 711.50 MiB free; 9.66 GiB reserved in total by PyTorch)

[EricHuiK]

[GutlapalliNikhil]

Found any solution for it? like Do we need to change any prams to solve it?

[hieunm1821]

I think you should reduce the batch-size

[NikhilChowdary-MCW]

@hieunm1821 , Yeah, we can reduce batch size or training resolution. Both cases will work.

[Mps24-7uk]

@EricHuiK Do you get solution

[A-Kerim]

Go to the .yaml in experiments/[dataset name]/..yaml file and update "BATCH_SIZE_PER_GPU" to a lower value. Then, run it as python -m torch.distributed.launch --nproc_per_node=1 tools/train.py ...

[Arshadoid]

Go to the .yaml in experiments/[dataset name]/..yaml file and update "BATCH_SIZE_PER_GPU" to a lower value. Then, run it as python -m torch.distributed.launch --nproc_per_node=1 tools/train.py ...

Hi @A-Kerim, I'm trying to run training on a single GPU on Windows 11 (just to see if it's running) and getting OutOfMemoryError. I already reduced the size of batch to 2. Still getting the same error. Would you have any suggestions how to solve this? Thanks!

[GutlapalliNikhil]

@Arshadoid Give it a try with batch size 1.

[Arshadoid]

@Arshadoid Give it a try with batch size 1.

Hi @GutlapalliNikhil thanks for the suggestion. I get an error about BatchNorm when try size of 1.