Retrieval-based-Voice-Conversion-WebUI icon indicating copy to clipboard operation
Retrieval-based-Voice-Conversion-WebUI copied to clipboard

Published 20 hours ago • verified publisher icon RVC-Project

CUDA out of memory

Open AlexNikMak opened this issue 1 year ago • 1 comments

Hi I am trying to train RVC(rmvpe-gpu) model using several gpus: rtx3070 8gb quadro m2000 4gb quadro m2000 4gb

I get this error: torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 2; 4.00 GiB total capacity; 3.21 GiB already allocated; 0 bytes free; 3.49 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF It's very strange because I can use this gpus for training, but separately.

My batch size is already at 1. Please guide me where I can manually adjust max_split_size_MB.

AlexNikMak avatar Feb 02 '24 09:02 AlexNikMak

You can use single GPU but with larger NVRAM size. Nvidia SLI is not currently supported.

ydlmz avatar Mar 29 '24 19:03 ydlmz