embeddedllm icon indicating copy to clipboard operation
embeddedllm copied to clipboard

[FEAT] Support OpenVINO NPU device

Open szeyu opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe.

Based on Intel OpenVINO docs : Run LLMs with OpenVINO GenAI Flavor on NPU

Now that NPU is available on OpenVINO, not just CPU and GPU

Describe the solution you'd like

Add npu as an option for device run by OpenVINO engine

szeyu avatar Sep 12 '24 03:09 szeyu