[Bug]: Error during Shape Inference of Multiply Node in OpenVINO Conversion of Torch Model with BatchNorm1d

[Thrsu]

OpenVINO Version

2024.0.0-14509-34caeefd078-releases/2024/0

Operating System

Ubuntu 18.04 (LTS)

Device used for inference

CPU

Framework

PyTorch

Model used

Given in the following script

Issue description

When attempting to convert and compile a Torch model using OpenVINO, an error occurs during shape inference of the Multiply node in the aten::batch_norm/BatchNormInference operation.

Step-by-step reproduction

1. Execute the following script：

import torch
from torch.nn import Module
import openvino as ov
import numpy as np


def compile_torch(model, input_data):
    ov_model = ov.convert_model(model, example_input=input_data)
    ir_path = f"temp_OVIR.xml"
    ov.save_model(ov_model, ir_path, compress_to_fp16=False)
    core = ov.Core()
    model = core.read_model(ir_path)

    compiled_model = core.compile_model(model=model, device_name="CPU")
    output_key = compiled_model.output(0)
    result = compiled_model(input_data)[output_key]
    return result

input_data = torch.randn([20, 99, 0], dtype=torch.float32)

torch_model = torch.nn.BatchNorm1d(100,affine=True,).eval()
torch_outputs = torch_model(input_data).cpu().detach().numpy()

trace = torch.jit.trace(torch_model, input_data)
trace = torch.jit.freeze(trace)

input_shapes = input_data.shape
res_ov = compile_torch(trace, input_data)
np.testing.assert_allclose(torch_outputs, res_ov, rtol=1e-3, atol=1e-3)

Relevant log output

RuntimeError: Exception from src/inference/src/cpp/infer_request.cpp:223:
Exception from src/plugins/intel_cpu/src/node.cpp:1587:
Shape inference of Multiply node with name aten::batch_norm/BatchNormInference failed: Exception from src/plugins/intel_cpu/src/shape_inference/custom/eltwise.cpp:45:
Eltwise shape infer input shapes dim index: 1 mismatch

Issue submission checklist

• [X] I'm reporting an issue. It's not a question.
• [X] I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• [X] There is reproducer code and related data files such as images, videos, models, etc.

[mvafin]

Is it expected that empty tensor is used as input? input_data = torch.randn([20, 99, 0], dtype=torch.float32) Your code will work for input: input_data = torch.randn([20, 100, 1], dtype=torch.float32)

[andrei-kochin]

@Thrsu have you tried the proposal suggested? Can we close the issue?

[avitial]

Closing this, I hope previous responses were sufficient to help you proceed. Feel free to reopen and ask additional questions related to this topic.