openvino_backend
openvino_backend copied to clipboard
triton-inference-server
OpenVINO backend for Triton.
This is a placeholder for the task that will enable usage of Intel GPU in Triton via OpenVINO.
Hi Thanks a lot for providing this backend. I have tried to use it and I have had some trouble getting Triton to load and run my OpenVINO models. I...
Hi, the readme states `OpenVINO backend currently supports inference only on Intel CPU devices using [OpenVINO CPU plugin](https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_supported_plugins_CPU.html)`. Does this mean when I am running the triton server, OpenVINO backend...
By using this model from Intel : https://docs.openvino.ai/2024/omz_models_model_age_gender_recognition_retail_0013.html I can't get good results (Or this model offers really good accuracy in the demo)... Instructions to replicate: Download model repo (it...
This is updated version of https://github.com/triton-inference-server/openvino_backend/pull/74 which doesn't require any changed on the base triton image. It gives option to specify TARGET_DEVICE to be passed to the OpenVINO Runtime during...
Code owners are automatically requested for review when someone opens a pull request that modifies code that they own. Code owners are not automatically requested to review draft pull requests....
