Fine-Tuning Guide to Add a New Speaker

[harshvardhan-truefan]

Hi, Really love the work done in this repo, it has been really helpful. Just a request, could you please add more documentation regarding fine-tuning the models for a new voice, using available model checkpoints. It is not very clear about how to fine-tune the model on a new dataset.

Thanks in advance!

Regards, Harsh

[ShyamGadde]

+1

[ultralegendary]

+1

[h2210316651]

I'd like one too, I haven't explored the codebase yet but I think it's based in coqui TTS. Hence similar training and fine-tuning methods would apply is my guess. There are some resources online detailing how to add a new speaker for coqui. I'll explore further and keep you guys posted.

[sachin7695]

@h2210316651 did you try to figureout the finetuning of indic tts?

[h2210316651]

@sachin7695 I have explored coqui in depth and decided it's way too complex for me to be using it. I have switched to rvc V2 I just TTS the content I want synthesized, then i use rvc to change the speaker voice, all you need is a 15 minute sample for pretty good quality voice clone.

[h2210316651]

@sachin7695 there's another project called applio, please check

[sachin7695]

@h2210316651 i tried xttsv2 coqui for fine tuning on hindi language and i was able to do that, i explored that in depth but the thing is the fastpitch of indic tts and coqui tts fastpitch is somewhat different when i checked the model state thats why i am pretty intersted to know indic tts fine tuning. anyways thanks a lot for your reply.

[sachin7695]

@h2210316651 bro i finetuned coqui xttsv2 for different indic language such as odia, bangla if at all you want to discover just ping me.. you just need to train the tokenizer (Byte pair encoding model) with indic transcription

[shrey802]

@sachin7695 any resources on how to fine tune xttsv2 coqui that I should follow to finetune for indic languages single speaker. Also have you tried fine tuning indic TTS?

[sachin7695]

@shrey802 indicTTS finetuning code is not available i am confused in the training of fastPitch of indicTTS did not get any resource for that i tried to finetune fastpitch model separately and integrate with indicTTS but encountered some errors!! you can ping me on linkedin (attached in my github profile) if you want assistance in fine tuning xttsv2 for indic languages