ipex-llm icon indicating copy to clipboard operation
ipex-llm copied to clipboard

[Nano] improve OpenVINO user experience

Open zhentaocc opened this issue 1 year ago • 0 comments

Description

bigdl.nano.openvino.OpenVINOModel has now provided limited features for users and few notifications about current status. We can only simply run inference like:

from bigdl.nano.openvino import OpenVINOModel
openvino_model = OpenVINOModel("resnet18-xnor-binary-onnx-0001.xml")
preds = openvnino_model.forward_step(x)

This is the only thing we can do right now. Here we can discuss about more features we can provide to improve user experience

Device setting

Similar to Pytorch, we can provide a function for users to simply copy models from one device to another.

openvino_cpu_model = OpenVINOModel("resnet18-xnor-binary-onnx-0001.xml", device='CPU')
openvino_gpu_model = openvino_cpu_model.to('GPU')
preds = openvnino_model.forward_step(x)

Notification/Log

Notify users of model information.

openvino_model = OpenVINOModel("resnet18-xnor-binary-onnx-0001.xml")

will output:

[INFO] File Path : resnet18-xnor-binary-onnx-0001.xml
[INFO] Precision: FP32
[INFO] Graph has 1 input and 2 outputs.
[INFO] Device: CPU

Extensions

Add extensions to OpenVINO Core. Seems if we want to support more devices, we need to load extra extensions. For example, NCS needs extra extension.
Needs further investigation.

zhentaocc avatar Jul 26 '22 02:07 zhentaocc