how to use vllm or trt improve inference?

[zbbwss]

Self Checks

• [x] I have thoroughly reviewed the project documentation (installation, training, inference) but couldn't find any relevant information that meets my needs. English 中文 日本語 Portuguese (Brazil)
• [x] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [x] Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell us your story.

how to use vllm or trt improve inference?

2. What is your suggested solution?

how to use vllm or trt improve inference?

3. Additional context or comments

No response

4. Can you help us with this feature?

• [ ] I am interested in contributing to this feature.

[PoTaTo-Mika]

Not supported in open-source repo.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.