Add a documentation page for data quality required for fine-tuning

[Aml-Hassan-Abd-El-hamid]

Self Checks

• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [X] Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

I'm trying to fine-tune the model to be able to pronounce Egyptian dialect.

I currently have a number of long videos -between 6 to 8 hours- that contain Egyptian books and the corresponding audio for different people reading those books, I'm cutting those audios into segments on silence and matching the segments to the text from the books, but I'm lacking some information to do so, such as:

1. How long should the ideal audio/text segments be to get the best results?
2. Should I keep the audio stereo or should I turn it to the mono channel?
3. Should I resample those audios or keep their original frequency?
4. should I delete the audio segments with slight background music or should I keep them?
5. should I keep the punctuation in the text or should I delete them?
6. Is there any cleaning for the text or the audio that should be done before fine-tuning?

2. Additional context or comments

No response

3. Can you help us with this feature?

• [X] I am interested in contributing to this feature.

[PoTaTo-Mika]

In the later version, we plan to remove the fine-tune part. Instead, we'll add a series of tools to enhance your reference audio's quality.

[Aml-Hassan-Abd-El-hamid]

But what if I need to add a new language or a dialect that the model usually doesn't handle? We need to fine-tune the model to accomplish such a task, right?

[PoTaTo-Mika]

But what if I need to add a new language or a dialect that the model usually doesn't handle? We need to fine-tune the model to accomplish such a task, right?

True, If you want to fine-tune for a new language (though the next version will support most of spoken languages in the world) ,you may need about 2K hours of low quality data, and about 100h (the more, the better) high quality data (44.1khz with high accuracy label). Hope that this can help you with running the project.

[Aml-Hassan-Abd-El-hamid]

Thanks a lot for your response, that's really helpful, I have one last question: does the data need to be cut to a certain length? I have multiple long audios -around 7 to 8 hours each- should I cut them down to shorter segments? and If I should do so, what is the recommended segment length? 15 minutes? 5 minutes? 30 seconds?

[PoTaTo-Mika]

Thanks a lot for your response, that's really helpful, I have one last question: does the data need to be cut to a certain length? I have multiple long audios -around 7 to 8 hours each- should I cut them down to shorter segments? and If I should do so, what is the recommended segment length? 15 minutes? 5 minutes? 30 seconds?

Yes, we recommend you to cut them into 30s / per segment.

[Aml-Hassan-Abd-El-hamid]

Thank you very much for your helpful and fast responses

[GalenMarek14]

But what if I need to add a new language or a dialect that the model usually doesn't handle? We need to fine-tune the model to accomplish such a task, right?

True, If you want to fine-tune for a new language (though the next version will support most of spoken languages in the world) ,you may need about 2K hours of low quality data, and about 100h (the more, the better) high quality data (44.1khz with high accuracy label). Hope that this can help you with running the project.

Thank you for your hard work on this project I was wondering if it's possible to provide a rough estimate for when the next model might be available? Even a ballpark estimate would be greatly appreciated.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[MithrilMan]

But what if I need to add a new language or a dialect that the model usually doesn't handle? We need to fine-tune the model to accomplish such a task, right?

True, If you want to fine-tune for a new language (though the next version will support most of spoken languages in the world) ,you may need about 2K hours of low quality data, and about 100h (the more, the better) high quality data (44.1khz with high accuracy label). Hope that this can help you with running the project.

@PoTaTo-Mika

do you have any ETA about the new version you are talking about? I'm very interested into Italian I'm actually trying to finetune f5-TTS but this is my first attempt at training a model and above all I don't have the resources ($$$) to train enough, I hope your new version includes Italian :)

[PoTaTo-Mika]

The model has supported Italian in the latest version (v1.5) , you can find the weights here: https://huggingface.co/fishaudio/fish-speech-1.5

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.