openvino_notebooks

openvino_notebooks copied to clipboard

OpenVINO model inference significantly slower than ONNX model

1

Hi, I converted a PyTorch .pt model to both OpenVINO and ONNX formats using the ultralytics package. Conversion code for OpenVINO: ```from ultralytics import YOLO def load_model(model_path): session = YOLO(model_path)...

rathoreaniket007

catvton notebook fails to safely seirilize

2

i installed the requirments in the main directory and proceeeded to install the notebook requirments restart the jupyter kernel and run. in the last cell of the notebook what I...

nevakrien

Native ReAct Agent can't run on NPU channel-wise model

2

**Describe the bug** sample https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/llm-agent-react/llm-agent-react.ipynb I converted a model for NPU. (Channel-wise quantization) `optimum-cli export openvino -m Qwen/Qwen2.5-3B-Instruct --weight-format int4 --sym --ratio 1.0 --group-size -1 Qwen2.5-3B-Instruct_NPU` Then, tried to run...

JamieVC

deepseek R1 distill Qwen 1.5B INT4-NPU or FP16

2

Hello, I am using openvino notebooks https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/deepseek-r1/deepseek-r1.ipynb on NPU Lunar Lake, core ultra 7 with 32GB of Memory in Windows 11 The prompt is click provided by gradio, after the...

susantowijaya

Unable to run qwen2-vl inference on Intel integrated GPU. Works fine with CPU

12

Hi, I'm unable to run inference on Intel Integrated GPU on qwen2-vl-2B model. It works fine if I select CPU as the device. The exception happens with error code "-5"...

paks100

Added notebook to run DeepSeek-R1-Llama-8B

2

guybd

[Torch FX][SD3] Add Dynamic Shapes

6

Add Dynamic shapes to export of transformer model in SD3 Torch FX notebook.

anzr299

Dynamic speculative decoding is significantly slower than auto-regressive and than speculative decoding generation

4

Below are my results when running speculative-sampling notebook. **Device:** GPU **Models and drafts:** **Phi-3 pair:** draft_model_id = "OpenVINO/Phi-3-mini-FastDraft-50M-int8-ov" target_model_id = "OpenVINO/Phi-3-mini-4k-instruct-int4-ov" **Llama3.1 pair:** draft_model_id = "OpenVINO/Llama-3.1-8B-Instruct-FastDraft-150M-int8-ov" target_model_id = "fakezeta/Meta-Llama-3.1-8B-Instruct-ov-int4" **Results:**...

shira-g

[Feature Request] CogAgent-9B OpenVINO support

3

Hi , I want to run this model (https://github.com/THUDM/CogAgent) in my computer (Intel NPU GPU), i'm not sure if OpenVINO already support this model , Thank you!

sanbuphy

Instant-id return black image and report invalid value encountered in cast

5

**Describe the bug** When I executed this jupyter it reports "[C:\Users\tester\Desktop\test\openvino_notebooks\openvino_env\Lib\site-packages\diffusers\image_processor.py:147](file:///C:/Users/tester/Desktop/test/openvino_notebooks/openvino_env/Lib/site-packages/diffusers/image_processor.py#line=146): RuntimeWarning: invalid value encountered in cast images = (images * 255).round().astype("uint8")" and return the black image **Expected behavior** A...

dannyweng88122