sglang icon indicating copy to clipboard operation
sglang copied to clipboard

[Feature] [Eagle] Are there any plans to support the feature for batching prefill?

Open BucherLi opened this issue 4 days ago • 1 comments

Checklist

  • [x] 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
  • [x] 2. Please use English, otherwise it will be closed.

Motivation

Now, I find the qps of sglang is very lower than baseline when use Eagle, I think the reason for this problem maybe is that sglang doesn't support batch prefill yet. So Do you have any plans to support this feature?

Related resources

Image No response

BucherLi avatar Feb 21 '25 02:02 BucherLi