vllm
vllm copied to clipboard
[Tracking issue] [Help wanted]: Deprecate BlockManagerV1
Anything you want to discuss about vllm.
We recently refactored the block allocation and management subsystem to improve its testability (PR https://github.com/vllm-project/vllm/pull/3492). We can replace the old implementation once the V2 is feature-complete and performant. This is a tracking issue for the remaining work.
Missing items:
- Prefix caching https://github.com/vllm-project/vllm/issues/3667
- CPU swapping https://github.com/vllm-project/vllm/issues/3666
- Sliding window support https://github.com/vllm-project/vllm/issues/3665
- Profile and optimize block manager V2 https://github.com/vllm-project/vllm/issues/4536