[Suggestion] Incoporate faster version chatterbox

[rikuddo91]

https://github.com/rsxdalv/chatterbox/tree/fast-with-top-p

Reddit thread: https://old.reddit.com/r/LocalLLaMA/comments/1lfnn7b/optimized_chatterbox_tts_up_to_24x_nonbatched/

This seems to be much faster at processing compared to regular Chatterbox version, when you get a chance, can you test how does it work with your fork? It would be amazing if the speed could be double or triple just by replacing the chatterbox version used.

[petermg]

Thanks. I'll check it out!

[C00reNUT]

that would be great! Ether the improved for or the recent vllm version - https://github.com/randombk/chatterbox-vllm - which even would be easier to implement.

[rikuddo91]

that would be great! Ether the improved for or the recent vllm version - https://github.com/randombk/chatterbox-vllm - which even would be easier to implement.

that looks amazing. It will be fantastic if those speeds are achievable with this Extended fork.

[petermg]

I should be able to do it I just need to find the time which I don't have but maybe next week.

[dwang3142]

any update on upgrading for faster inference? thanks! @petermg

[petermg]

any update on upgrading for faster inference? thanks! @petermg

I did spend some time on it, like a few hours and then it was getting to have different issues and I didn't have more time to try from different approaches, but it IS still on my mental to-do list.