vllm icon indicating copy to clipboard operation
vllm copied to clipboard

[Tracking issue] [Help wanted]: Deprecate BlockManagerV1

Open cadedaniel opened this issue 1 year ago • 0 comments

Anything you want to discuss about vllm.

We recently refactored the block allocation and management subsystem to improve its testability (PR https://github.com/vllm-project/vllm/pull/3492). We can replace the old implementation once the V2 is feature-complete and performant. This is a tracking issue for the remaining work.

Missing items:

  • Prefix caching https://github.com/vllm-project/vllm/issues/3667
  • CPU swapping https://github.com/vllm-project/vllm/issues/3666
  • Sliding window support https://github.com/vllm-project/vllm/issues/3665
  • Profile and optimize block manager V2 https://github.com/vllm-project/vllm/issues/4536

cadedaniel avatar May 01 '24 18:05 cadedaniel