Inference slow

[ChengsongLu]

Self Checks

• [X] I have thoroughly reviewed the project documentation (installation, training, inference) but couldn't find any relevant information that meets my needs. English 中文 日本語 Portuguese (Brazil)
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [X] Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell us your story.

python -m tools.run_webui
--llama-checkpoint-path "./ckpt/fish-speech-1.5"
--decoder-checkpoint-path "./ckpt/fish-speech-1.5/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"
--decoder-config-name firefly_gan_vq

without "--compile", I got ~60 tokens/sec

with "--compile", I met error: "W1210 18:19:53.543000 139724024968768 torch/fx/experimental/symbolic_shapes.py:4449] [0/1] xindex is not in var_ranges, defaulting to unknown range."

Besides, this error doesn't happen when I am using: python tools/llama/generate.py
--text "The text you want to convert"
--prompt-text "Your reference text"
--prompt-tokens "fake.npy"
--checkpoint-path "checkpoints/fish-speech-1.5"
--num-samples 2
--compile

But it just don't speed up.

2. What is your suggested solution?

None

3. Additional context or comments

No response

4. Can you help us with this feature?

• [X] I am interested in contributing to this feature.