MeloTTS icon indicating copy to clipboard operation
MeloTTS copied to clipboard
verified publisher icon myshell-ai

inference time on cpu

Open FollowingT opened this issue 9 months ago • 2 comments

Inferencing a sentence with 10 words cost 12s on average on CPU. Any idea to improve the inference time on cpu?

FollowingT avatar Mar 14 '25 08:03 FollowingT

hi @FollowingT, did you considered run inference on integrated GPU? I have created this PR to enable it, maybe you can take a look.

Desmond0804 avatar Mar 17 '25 07:03 Desmond0804

It is because this is VITS architecture—but the bottleneck is BERT encoder (read the code). Encoding is slow + yeah, try using clear VITS and see if it improves (it does)

yukiarimo avatar Aug 01 '25 06:08 yukiarimo