markusbr

OpenVINO uses its own format, Intermediate-Representation (IR). OpenVINO supports various other formats like ONNX. Once a model (e.g. in ONNX-format) was loaded (and compiled) it can be exported - and...

You can find a "mobilenet-ssd" model on the new model-zoo under https://models.luxonis.com/luxonis/mobilenet-ssd/2da6e0a5-4785-488d-8cf5-c35f7ec1a1ed?backTo=%2F%3Fsearch%3Dmobilenet , which is a `mobilenet-ssd-300x300_8shave.blob`. When downloading the model you can find a `buildinfo.json` in the downloaded archive,...

What actually is your question? Would you mind sharing your code to see what the community could help you with?

There is already a note at the top of the README: > Important > bigdl-llm has now become ipex-llm, and our future development will move to the [IPEX-LLM](https://github.com/intel-analytics/ipex-llm) project. Looks...

Can yo uprovide more information, please? What does your environment look like, which SoC/CPU/GPU do you have? Which Python version? Do you use a Python virtual environment? Which version of...

Are you talking about "https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-chatbot" (where `llama-3-8b-instruct` is mentioned) or "https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-question-answering" (where e.g. `tiny-llama-1b-chat` is mentioned)? Can you provide more details about your system, please (SoC, amount of memory, OS,...

Have you seen errors or warnings in the steps for conversion and compression? Do you see the same when using the INT8 or FP16 variant instead of the INT4 variant?...

For conversion and compression I would anyway expect the OperatingSystem to start swapping memory to HDD/SSD if the system memory is not big enough... Let's see if someone else can...

Can you clean the build directory and trigger `cmake` (and show your finally used command line with enabled options) and share the console output? Looks like one of the thirdparties...

Can you provide more details, please?