Memory Problem

[TristanYoshi]

Hello I am a newbie and I'm using this code to render visuals for integration in a movie sequence. I'm new to coding or python, not experienced at all really, and I got this error:

RuntimeError: CUDA out of memory. Tried to allocate 188.00 MiB (GPU 0; 6.00 GiB total capacity; 4.29 GiB already allocated; 0 bytes free; 4.74 GiB reserved in total by PyTorch)

I was wondering if you know what I could do to fix this problem besides getting a new graphics card or using a different computer.

[Aniket27100709]

import torch torch.cuda.empty_cache() provides a good alternative for clearing the occupied cuda memory and we can also manually clear the not in use variables by using,

import gc
del variables
gc.collect()

But still after using these commands, the error might appear again because pytorch doesn't actually clears the memory instead clears the reference to the memory occupied by the variables. So reducing the batch_size after restarting the kernel and finding the optimum batch_size is the best possible option (but sometimes not a very feasible one).

Another way to get a deeper insight into the alloaction of memory in gpu is to use:

torch.cuda.memory_summary(device=None, abbreviated=False)

wherein, both the arguments are optional. This gives a readable summary of memory allocation and allows you to figure the reason of CUDA running out of memory and restart the kernel to avoid the error from happening again (Just like I did in my case).

Passing the data iteratively might help but changing the size of layers of your network or breaking them down would also prove effective (as sometimes the model also occupies a significant memory for example, while doing transfer learning).

[TristanYoshi]

Thank you for getting back to me with this help. Sorry I didn't respond sooner! I ended up using a lower resolution 2.5k frame instead of 4k for this. I have been rendering non stop in hopes to complete this acid trip sequence and thus the entire movie in time for it's premiere at my local Alamo Drafthouse Cinema in under a month!

I'm just about to start experimenting with the ImageNet classes to put a little meaning into the visuals land having them relate to what's going thru the character's minds while on the drugs. I was wondering - is there a way in the config to make the visuals have more variance like more random shapes or the shapes moving more randomly.?