Woosuk Kwon

[SpecDecode] Support EAGLE in V1

14

- [x] 1. Correctly initializing and loading the EAGLE draft model - [x] 2. Consider the lookahead slots in the KV cache manager - [x] 3. Cache `draft_probs` inside the...

[RFC]: Deprecating vLLM V0

26

vLLM V1 has been the default engine since version v0.8.0, released approximately three months ago. With substantial user adoption and overwhelmingly positive feedback on V1, we propose formally deprecating vLLM...

[Multimodal] Optimize Qwen2/2.5-VL startup time

4

## Essential Elements of an Effective PR Description Checklist - [ ] The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)". -...

[Do not merge] Cache model info for faster startup

1

A hacky way to save ~6 secs in startup time.

GPU Model Runner V2

8

# Key Changes * Remove persistent batch * No “reordering” & complex bookkeeping * Almost all CPU states are Numpy arrays → We can vectorize most of the Python loops...