Zero-Shot Voice Cloning

[fakerybakery]

Hi, I know this library is primarily for text -> voice but do you know if it would be possible to modify it to accept a speaker embedding and perform zero-shot voice cloning? Thanks!

[digisomni]

Yep, this is what I am after as well. Bark did this, if we have something like that then using this for an assistant becomes 10x easier since any personality can be inserted into it.

[ylacombe]

Hey @fakerybakery, thanks for opening the discussion! The current design is a choice, and we're currently discussing internally if adding zero-shot voice cloning makes sense!

[lheuveline]

Hey @fakerybakery, thanks for opening the discussion! The current design is a choice, and we're currently discussing internally if adding zero-shot voice cloning makes sense!

Hey ! Congrats for your really impressive model, I'm really happy and enthousiastic to see HF finally getting into TTS field :D The output quality for 10k hours of training is really good. Concerning zero-shot voice cloning, even if you agree to add this feature to the current architecture, the model will need to be train again frm scratch no ?

[johnwick123f]

+1, it would be very useful for many things. Parler tts sounds very good and it would be great to support cloning voices

[sanchit-gandhi]

Hey all - the two fine-tuned checkpoints (Jenny and Expresso) are interesting intermediate artefacts while we wait for the next pre-trained version of the model with consistent voice support

[mantrakp04]

I got really good results with zero shot voice cloning using myshell ai's openvoice 2, m also willing to support in any way to make parler tts multi lingual, lmk