latent-nerf icon indicating copy to clipboard operation
latent-nerf copied to clipboard

Published 20 hours ago • verified publisher icon eladrich

CUDA out of memory issue

Open mshooter opened this issue 2 years ago • 3 comments

Hi, I keep getting Cuda out of memory... Which parameters should I change for this?

mshooter avatar Mar 16 '23 11:03 mshooter

I have the same issue...

flow-specter avatar May 11 '23 11:05 flow-specter

if you met this issue: RuntimeError: CUDA error: an illegal memory access was encountered

You can simply try to re-run the training script repeatedly (like 4,5 times) . It worked for me.

qsong2001 avatar May 11 '23 13:05 qsong2001

Same issue on 4GB RAM. torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 26.00 MiB (GPU 0; 4.00 GiB total capacity; 3.38 GiB already allocated; 0 bytes free; 3.42 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF I tried different solutions such as reducing batch size, using torch.no_grad() for Inference and loading models in float16 precision instead of the default float32 precision; But none of them worked for me.

What is the minimum hardware requirements?

SzaremehrjardiMT avatar Mar 07 '24 18:03 SzaremehrjardiMT