vllm-project
[RFC]: Drop Support for OpenVINO
Motivation.
OpenVINO backend was initially integrated as an alternatively to the CPU backend and has branched out the vLLM execution logic for every levels (executor, model runner, and attention backend). #5377
Over the last 9 months, we have been the following
- Relatively low usage as reported in Github Issues and Slack discussions
- The Intel CPU codepath is more mature and largely compatible for Arm as well.
- The OpenVINO code path complicated with codebase
- CI and build became difficult to maintain
I would like to propose to move OpenVINO off from the main codebase, and transition to a vLLM out of tree platform plugin if desired. OpenVINO can follow the same approach as Ascend and Spyre with the plugin approach #11162
Proposed Change.
- Remove OpenVINO codepath, build and test.
- Optionally, create vllm-project/vllm-openvino if the developers want to maintain plugin level compatibility.
Feedback Period.
2 weeks. By March 20.
CC List.
cc @ilya-lavrenov @WoosukKwon @youkaichao @robertgshaw2-redhat @mgoin
Any Other Things.
No response
Before submitting a new issue...
- [x] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.
agree that a plugin might be better for openvino, similar to https://github.com/vllm-project/vllm-spyre and https://github.com/vllm-project/vllm-ascend
https://github.com/vllm-project/vllm-openvino has been created. We will move forward with the removal.
