Meaningless sounds get produced randomly if there are continuous spaces(newlines, tabs) regardless of prompts or input content

[AndyRightNow]

Self Checks

• [X] This template is only for bug reports. For questions, please visit Discussions.
• [X] I have thoroughly reviewed the project documentation (installation, training, inference) but couldn't find information to solve my problem. English 中文 日本語 Portuguese (Brazil)
• [X] I have searched for existing issues, including closed ones. Search issues
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [X] Please do not modify this template and fill in all required fields.

Cloud or Self Hosted

Self Hosted (Source)

Environment Details

Ubuntu 22.04.3 LTS (Google Colab), Python 3.10, torch==2.4.1

Steps to Reproduce

Run tools/llama/generate.py with any Chinese text that has continuous whitespaces, such as indentations, and with any prompts.

For example:

    要转换的文本。
    要转换的文本？
    要转换的文本。
    “要转换的文本！”
    要转换的文本。

Note that if the indents at the beginning each line get removed, the random meaningless sounds also disappear. Also the meaningless sounds don't always appear on every line. It happens periodically and has no specific patterns.

Right now my workaround is to add an additional clean-up step here inside clean_text https://github.com/fishaudio/fish-speech/blob/f54c50fe6239d252be88a72433580fdb4118d350/fish_speech/text/clean.py#L24 to replace any continuous whitespace characters with the first occurrence of the matched characters (including newlines or tabs).

✔️ Expected Behavior

No random meaningless sounds.

❌ Actual Behavior

As described above.

[builtbybasit]

+1

[talaon2]

+1

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.