neuralmagic
support for converting quantized onnx model to blob
Hello, I am keen to convert my quantized trained ONNX model into a blob file. OpenVino currently does not support this which is what I've been using so far. Is there any plans to add support for this or model conversion features? It's a long shot I know but just thought I'd try. Thanks in advance :)
Hi @d-smit could you share a link to what this blob file format is like? ONNX is already a binary format so is it a matter of compression for you? DeepSparse does take the ONNX and compress it down to an internal representation specific to the available hardware. Hope this is helpful
The purpose of converting to the blob is for deployment on the OAK-D camera device, as expected by the DepthAI spatial AI API used for working with OAK.
Info here: https://docs.luxonis.com/en/latest/pages/tutorials/deploying-custom-model/ Roughly following this tutorial: https://blog.roboflow.com/luxonis-oak-d-custom-model/
It has been working training YOLO models with Ultralyics, exporting to ONNX then converting to OpenVINO IR and then converting to a blob. Just not when trying with the sparse quantized YOLO model ONNX files
Okay, thanks for pointing us to the right docs @d-smit . It looks like that OAK-D camera device uses a custom hardware accelerator called MyriadX VPU Processor. DeepSparse focuses on running fast inference on CPUs only so it doesn't support this accelerator. It looks like Intel makes this MyriadX VPU so it would make sense that OpenVINO is the software to use it. Not sure if we can help much with OpenVINO ONNX support, sorry.
Closing this issue as it seems the hardware is proprietary to Intel. Feel free to respond or reopen if you have more questions, thanks!