sglang
sglang copied to clipboard
[Feature] [Eagle] Are there any plans to support the feature for batching prefill?
Checklist
- [x] 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
- [x] 2. Please use English, otherwise it will be closed.
Motivation
Now, I find the qps of sglang is very lower than baseline when use Eagle, I think the reason for this problem maybe is that sglang doesn't support batch prefill yet. So Do you have any plans to support this feature?
Related resources
No response