[example]Add new super-resolution model from OpenVINO model zoo

[NALLEIN]

Add new super-resolution model from openvino. Here is the corresponding information of the model :

• single-image-super-resolution-1032
• single-image-super-resolution-1032 Model Performance In OpenVINO
• single-image-super-resolution-1032
• single-image-super-resolution-1033 Model Performance In OpenVINO
• Paper These two models can be run in OpenVINO and the performance of these two models is

Model	OpenVINO Plugin	Inference Time (ms)
single-image-super-resolution-1032	clDNN(UHD630)	249
single-image-super-resolution-1032	MKL-DNN(i7-10710U)	1352
single-image-super-resolution-1033	clDNN(UHD630)	247
single-image-super-resolution-1033	MKL-DNN(i7-10710U)	1224

I tried to run these two models in polyfill, but I encountered the following problems during the model compilation stage:

Error: Tensor 7:2 is not found
    at OpenVINOModelImporter._getTensorId (OpenVINOModelImporter.js:215)
    at OpenVINOModelImporter._addOpsAndParams (OpenVINOModelImporter.js:388)
    at OpenVINOModelImporter.createCompiledModel (OpenVINOModelImporter.js:49)
    at async BaseRunner.compileModel (BaseRunner.js:255)
    at async SuperResolutionExample._compileModel (BaseExample.js:394)
    at async SuperResolutionExample.main (BaseExample.js:509)

The reason is that the Tensor corresponding to the input cannot be found in the .bin file when analysing the eltwise operation . This problem can be reproduced in this repo. I only made small changes to OpenVINOModelImporter.js and the problem seems to be in the weight file or when OpenVINOModelImporter.js calling the _addTensorOperands function the corresponding Tensor was not resolved . Currently I can't locate the problem. I encountered this error in eltwise operation in both models，corresponding to the operation of name = "39" and the operation of name = "37" respectively.

[NALLEIN]

I fixed the problem that relu with multiple outputs may caused the following ops can't find corresponding input tensor. Relu needs to be executed seperately and there are continuous relu in this model. Besides , there are 6D tensors in this model I can not find a clear format description for 6D tensor. And I can just run the model in polyfill but cann't get right result due to these tensors. Do you have any suggestions on dealing with 6D tensor permute, and data reorder ?

[NALLEIN]

@huningxin Do you have any suggestions in this problem?

[ibelem]

@huningxin Any suggestions?

[huningxin]

Generally WebNN uses channel-last layout, OpenVINO uses channel-first layout. Could you please share what are the shapes (convolution, reshape, permute) when you run it with webml-polyfill?

[NALLEIN]

Generally WebNN uses channel-last layout, OpenVINO uses channel-first layout. Could you please share what are the shapes (convolution, reshape, permute) when you run it with webml-polyfill?

Yes, the tensor is shown in this figure. The model reshapes the channel dimension to 3 dimensions and then 3 separate channel dimensions was permuted.

[huningxin]

Is this figure just for openvino model? I suppose WebNN uses NHWC layout, e.g. so the shape of Convolution should be [1, 360, 640, 72], is it?

[NALLEIN]

Is this figure just for openvino model? I suppose WebNN uses NHWC layout, e.g. so the shape of Convolution should be [1, 360, 640, 72], is it? Yes, this is openvino model and the shape of conv output should be [1, 360, 640, 720] and reshape output should be [1, 360, 720, 8, 3, 3]. But I have no idea how to permute and I try to permute using order[0, 1, 4, 2, 5, 3] and [0, 4, 1, 5, 2, 3] but seems doesn't work.

[huningxin]

Did you try to permute [1, 360, 720, 8, 3, 3] to [1, 360, 3, 640, 3, 8], then reshape to [1, 1080, 1920, 8]?

[NALLEIN]

Did you try to permute [1, 360, 720, 8, 3, 3] to [1, 360, 3, 640, 3, 8], then reshape to [1, 1080, 1920, 8]?

Yes,I tried to permute [1, 360, 720, 8, 3, 3] to [1, 360, 3, 640, 3, 8], or[1, 3, 360, 3, 640, 8] then reshape to [1, 1080, 1920, 8].

[NALLEIN]

I replace rehspape->permute->reshape with depthToSpace op:

        case 'Reshape': {
          let nextNode = graph.nodes[i + 1];
          let next2Node = graph.nodes[i + 3];
          if(nextNode && next2Node && nextNode.operator === 'Permute' && 
          next2Node.operator === 'Reshape' )
          {
            const input = node.inputs[0];
            console.log(`input shape: [${input.shape()}]`);
            const blockSize  = node.outputs[0].shape()[4];
            inputs.push(this._getTensorId(input));
            inputs.push(this._addScalarInt32(blockSize));
            console.log(`blockSize ${blockSize}`);
            const output = next2Node.outputs[0];

            // Add outputs
            const outDims = output.shape();
            const outputType = {
              type: this._getTypeCode(output.dataType()), dimensions: outDims
            };
            const outputId = this._addNamedOperand(output.graphId(), outputType);
            outputs.push(outputId);
            console.log(`  output shape: [${outDims}]`);
            i += 3;
            console.log('Merge Reshape->Permute->Reshape into depthToSpace');
            opCode = this._nn.DEPTH_TO_SPACE;
          } 

This can bypass the processing of 6D tensor but still can not get the correct result, reasons may be:

• depthToSpace op cannot handle two const operations

• The model requires some pre-processing of mean and variance