[Bug]: When i use Ultra9 NPU to run models, it report: get_shape was called on a descriptor::Tensor with dynamic shape

[MansonHua]

OpenVINO Version

2024.1 Recommended Windows PIP includes NPU plugin Python API only

Operating System

Windows System

Device used for inference

NPU

Framework

PyTorch

Model used

cardiffnlp/twitter-roberta-base-sentiment-latest

Issue description

When i run huggingface models with openvino, just like the Tutorial did: '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' import numpy as np import torch from transformers import AutoModelForSequenceClassification, AutoTokenizer import openvino as ov from pathlib import Path

MODEL = "cardiffnlp/twitter-roberta-base-sentiment-latest"

tokenizer = AutoTokenizer.from_pretrained(MODEL, return_dict=True) model = AutoModelForSequenceClassification.from_pretrained(MODEL, torchscript=True)

text = "HF models run perfectly with OpenVINO!" encoded_input = tokenizer(text, return_tensors="pt", padding='max_length', max_length=128)

def print_prediction(scores): for i, descending_index in enumerate(scores.argsort()[::-1]): label = model.config.id2label[descending_index] score = np.round(float(scores[descending_index]), 4) print(f"{i+1}) {label} {score}")

save_model_path = Path("./models/model.xml")

if not save_model_path.exists(): ov_model = ov.convert_model(model, example_input=dict(encoded_input)) ov.save_model(ov_model, save_model_path)

core = ov.Core()

compiled_model = core.compile_model(save_model_path, 'NPU') input_data = {key: value.detach().numpy() for key, value in encoded_input.items()} scores_ov = compiled_model(input_data)[0] scores_ov = torch.softmax(torch.tensor(scores_ov[0]), dim=0).detach().numpy()

print_prediction(scores_ov) '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' I change the device from 'AUTO' to 'NPU', and the Error reports:

RuntimeError Traceback (most recent call last) Cell In[34], line 29 25 ov.save_model(ov_model, save_model_path) 27 core = ov.Core() ---> 29 compiled_model = core.compile_model(save_model_path, 'NPU') 30 input_data = {key: value.detach().numpy() for key, value in encoded_input.items()} 31 scores_ov = compiled_model(input_data)[0]

File ~\miniconda3\envs\inference\Lib\site-packages\openvino\runtime\ie_api.py:521, in Core.compile_model(self, model, device_name, config, weights) 516 if device_name is None: 517 return CompiledModel( 518 super().compile_model(model, {} if config is None else config), 519 ) 520 return CompiledModel( --> 521 super().compile_model(model, device_name, {} if config is None else config), 522 ) 523 else: 524 if device_name is None:

RuntimeError: Exception from src\inference\src\cpp\core.cpp:126: Exception from src\inference\src\dev\plugin.cpp:59: Exception from src\plugins\intel_npu\src\plugin\src\plugin.cpp:513: get_shape was called on a descriptor::Tensor with dynamic shape

Step-by-step reproduction

No response

Relevant log output

No response

Issue submission checklist

• [X] I'm reporting an issue. It's not a question.
• [X] I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• [X] There is reproducer code and related data files such as images, videos, models, etc.