"CUDA out of memory" when generating longer duration audio. Any way to fix?

[asudyn]

I'm an enthusiast trying to see what is possible with this model and I can generate 30 second pieces on my RTX 3060 just fine but when I try 60 seconds I get this:

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 6.75 GiB (GPU 0; 12.00 GiB total capacity; 10.34 GiB already allocated; 0 bytes free; 10.69 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Is it possible to generate longer audio without buying a new GPU?

[CounterFlow64]

30 seconds is already great for a 3060. But if you want, you can reduce the -n N_CANDIDATE_GEN_PER_TEXT to the lowest value, 1. From my understanding, this option generates a set number of candidates. So if it's set to 5, then the single generation comes from 5 attempts who all use up lots of memory, and it tries to pick the best attempt to output.

However, I have experimented with this. By setting it to 1, I cannot consistently hear a quality difference, plus it becomes MUCH faster and uses MUCH less memory.

[gelukuMLG]

The -n thing doesn't work I get the exact same oom issue with 3 and 1.