metavoice-src icon indicating copy to clipboard operation
metavoice-src copied to clipboard

Will it natively support other languages?

Open faceair opened this issue 1 year ago • 9 comments

I saw that the readme described the training data mainly in English, and I was worried that it would not learn the prosody in other languages, for example, the prosody in Chinese should be very different from English. Will it be possible to learn other languages in the future just by fine-tuning?

faceair avatar Feb 07 '24 03:02 faceair

Yes, please elaborate more on:

Support for (cross-lingual) voice cloning with finetuning. We have had success with as little as 1 minute training data for Indian speakers.

Can you please share the Indian speakers examples code and TTS results?

farside-sh avatar Feb 07 '24 03:02 farside-sh

Joining the chorus here! I would like to fine-tune this for Italian speech2speech. Could you release a tutorial on how to do this?

AndreaPi avatar Feb 07 '24 21:02 AndreaPi

I also want to raise this question, looking for spanish :)

The system looks fantastic.

juangea avatar Feb 08 '24 13:02 juangea

Will it be possible to learn other languages in the future just by fine-tuning?

@faceair / anyone else here for that matter - until we release the fine-tuning code, are you open to testing this hypothesis with a LoRA implementation? It would greatly benefit the community. Others can adopt it for different languages too :)

sidroopdaska avatar Feb 09 '24 01:02 sidroopdaska

Looking forward to multilingual support!

philpav avatar Feb 13 '24 21:02 philpav

If there is some info on how to train the Lora and how to apply it, I’m all for it, in the end a LORA can be trained in consumer hardware like a 4090, isn’t it?

juangea avatar Feb 14 '24 08:02 juangea

Will it be possible to learn other languages in the future just by fine-tuning?

@faceair / anyone else here for that matter - until we release the fine-tuning code, are you open to testing this hypothesis with a LoRA implementation? It would greatly benefit the community. Others can adopt it for different languages too :) I can try that.

MonojitBanerjee avatar Feb 18 '24 01:02 MonojitBanerjee

Will it be possible to learn other languages in the future just by fine-tuning?

@faceair / anyone else here for that matter - until we release the fine-tuning code, are you open to testing this hypothesis with a LoRA implementation? It would greatly benefit the community. Others can adopt it for different languages too :)

Sure, would be interested

bread-on-toast avatar Feb 19 '24 09:02 bread-on-toast

Please give us a few days, and we will share a reference implementation for this

sidroopdaska avatar Feb 19 '24 11:02 sidroopdaska

I've added some initial pointers to this here: https://github.com/metavoiceio/metavoice-src/issues/70#issuecomment-1957337895

vatsalaggarwal avatar Feb 21 '24 17:02 vatsalaggarwal

Would be interested for French LoRA here :)

maepopi avatar Mar 01 '24 20:03 maepopi

Hope to support Chinese.

mzdk100 avatar Mar 03 '24 13:03 mzdk100

I'm closing this issue in favour of https://github.com/metavoiceio/metavoice-src/issues/70 where active work seems to be happening!

vatsalaggarwal avatar Mar 04 '24 13:03 vatsalaggarwal