Slow issue. I put about 30 characters long a single sentence and it takes an hour.

[meeplindkvist]

I use Ryzen 5600 and RTX3070 and It takes 1~2 hours for a single sentence. My gpu load is only 10% and Cpu has 60-70%.

I have 32gb ram 8GB Vram and I haven't changed any code yet. I had some trouble while I'm installing it but it seems working with a very slow speed.

I heard from other people, they took about a minute~ 10 minutes. But not over an hour. I think someting is wrong with me.

Can you help me to solve this problem? Thank you!

[MaxLikesCode]

For me it takes 30 minutes for 5 words. Absolutely unusable in this state. Any idea? Edit: I have a Ryzen 3700x with a Nvidia Titan X Pascal.

[meeplindkvist]

I fixed problem by myself.

I found the issue that my pytorch has been uninstalled and re-installed while installing requirement modules. Problem was that re-installed pytorch wasn't cuda supporting. Therefore I uninstalled those packages

pip3 uninstall torch torchvision torchaudio

and re-installed pytorch with cuda supporting

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117

after this my vram is being used and it's quite much faster than before. At least not 1 hour for one sentence.

I also found a fork that is boosting speed

https://github.com/152334H/tortoise-tts-fast

[n8bot]

One very important tip that I just discovered, and I have not seen mentioned. The first step of the process Generating autoregressive samples... is the most compute-intensive part. the other steps Computing best candidates using CLVP and Transforming autoregressive outputs into audio... are much less demanding.

What is interesting, is that the number of candidates that do_tts.py outputs has no effect (it seems) on the length of time needed for that first demanding step.

So, to speed up iteration, if you want to have many options to choose from for a given sample, crank the candidates count up. This way you will spend less time re-generating the same prompt to try and get a better result.

This will save me hours of time in the long run.

[adunato]

I fixed problem by myself.

I found the issue that my pytorch has been uninstalled and re-installed while installing requirement modules. Problem was that re-installed pytorch wasn't cuda supporting. Therefore I uninstalled those packages

pip3 uninstall torch torchvision torchaudio

and re-installed pytorch with cuda supporting

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117

after this my vram is being used and it's quite much faster than before. At least not 1 hour for one sentence.

I also found a fork that is boosting speed

https://github.com/152334H/tortoise-tts-fast

Thanks for this, it was offloading to CPU because of this for me and if I had not read your comment I would have been clueless on how to fix it. I would also add to check your cuda toolkit version (nvcc - - version) as I think only 11.7 and 11.8 are supported by pytorch. The main issue is that these dependencies don't necessarily show up as errors, but it just doesn't work. Anyway, now my setup went from more than one hour to a minute 👍