Doesen't seem to be using GPU when generating samples. Extremely long render times. How do I fix this?

[embanot]

Sorry I'm a complete newbie with all this. I followed all the instructions online and through some online tutorials to get the tortoise running locally. I got it working so that I can actually generate samples, but the render times (if that's the appropriate term here) are extremely long (a few hours for just for a 2 sentance sample). When looking at my PC performance, it's very high on the CPU measure but practically nothing on the GPU. How do I get it to utilize my GPU instead? I'm using a 3070 TI so it is CUDA compatible.

A short list of what I've done:

• installed Anaconda3
• installed pytorch with cuda 11.6
• cloned the git
• installed all the dependancies
• installed pysoundfile

Sorry I am new to all of this so if someone could provide dumbed down version of what I need to do in order to get tortoise to run off my GPU, that would be greatly appreciated! Thank you!

[hkilian]

Hey,

I had the same problem and it was because pytorch was not actually installed with Cuda.

https://stackoverflow.com/questions/60987997/why-torch-cuda-is-available-returns-false-even-after-installing-pytorch-with

Check the top answer here and confirm that torch actually has CUDA enabled.

If its not then try running this: pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0+cu113 -f https://download.pytorch.org/whl/torch_stable.html

After that it worked for me on the GPU. Make sure to close the command prompt and reopen after installing pytorch again

[embanot]

thank you for the reply.

Where do I type in the >>>Import Torch ...etc in order to check if Cuda is available? I tried typing it in Anaconda Prompt and it didn't work . Am I supposed to be using something else to check?

Also I tried running the comand you suggested and now I get an error when I try to create a voice sample. It says "The procedure entry point could not be located in the dynamic link library. c:\ProgramData\Anaconda3\lib\site-packages\torchaudio\lib\libtorchaudio.pyd

I think I may just uninstall Anaconda and all the packages and start from fresh. would you be able to list out each command I need to enter on a fresh version of Anaconda?

[chaosdrop]

I'm super noob don't even have it working yet, but to test if you have working CUDA you'd make a python file like cudatest.py

import torch x = torch.cuda.is_available() print(x)

to it and then run that from Anacoda.

Effetely it's not a command prompt command it's a python script.

[embanot]

thanks. so i tested it out and it does infact say false. So Cuda isn't available. No idea what to do now haha.

[chaosdrop]

I got mine working just following the steps you listed, so odds are you probably just don't have a video card that supports the right version of CUDA or from a little bit of reading driver versions matters too, so if you haven't updated your drivers in a while that could be the issue.

If updating drivers doesn't work you'll probably have to look through the previous versions https://pytorch.org/get-started/previous-versions/

And figure out what versions of everything you need that'll actually work for your video card, or buy a new video card. I have a 4090 and CUDA worked out of the box with no extra steps beyond what you've already listed.

[hkilian]

I'm also on a 4090 which I'm guessing supports the latest cuda. Try installing torch with lower versions of cuda for the 3070

[embanot]

Thanks everyone. I managed to solve it by starting over and just following the same steps I outlined. Not sure why it worked this time around, but It's all good for me now. thanks!