ONE [nnpackage] Add multiple model nnpkg example

It adds two tflite models of nnpkg under nnpackage/examples. MANIFEST has pkg-inputs, pkg-outputs and model-connect fields.

ONE-DCO-1.0-Signed-off-by: Sanggyu Lee [email protected]

Related: #9116

Please don't merge this PR. I would like to finalize nnpackage spec v1.3.0 after seeing our new spec works (= compilation and execution).
I would like to use this example in positive test for multiple models as we discussed in https://github.com/Samsung/ONE/issues/9389#issuecomment-1174601169

Jul 05 '22 06:07 glistening

@hseok-oh I think #9129 assumes the multiple models are generated using circle-partitioner tool. On the other hand, I made this example in the way I described in https://github.com/Samsung/ONE/issues/9206#issuecomment-1148276287. I want to add the simplest example which does not depend on specific tools.

Jul 05 '22 07:07 glistening

How I made this example nnpkg

$ wget https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_02_22/mobilenet_v1_1.0_224.tgz
$ tar -zxf mobilenet_v1_1.0_224.tgz

$ python tools/tflitefile_tool/select_operator.py mobilenet_v1_1.0_224.tflite <( echo 0-1 ) mv1.0_1.tflite
$ python tools/tflitefile_tool/select_operator.py mv1.0_1.tflite <( echo 0 ) mv1.0.tflite
$ python tools/tflitefile_tool/select_operator.py mv1.0_1.tflite <( echo 1 ) mv1.1.tflite

# make sure three tflite is valid
$ ./Product/x86_64-linux.debug/out/bin/tflite_comparator mv1.0_1.tflite
$ ./Product/x86_64-linux.debug/out/bin/tflite_comparator mv1.0.tflite
$ ./Product/x86_64-linux.debug/out/bin/tflite_comparator mv1.1.tflite

$ tools/nnpackage_tool/model2nnpkg/model2nnpkg.sh -m mv1.0.tflite mv1.1.tflite -p two_tflites
$ cat two_tflites/metadata/MANIFEST
{
  "major-version" : "1",
  "minor-version" : "2",
  "patch-version" : "0",
  "configs"     : [  ],
  "models"      : [ "mv1.0.tflite", "mv1.1.tflite" ],
  "model-types" : [ "tflite", "tflite" ]
}

# update minor-version, and add additional fields manually

Jul 06 '22 04:07 glistening

@chunseoklee You can use this nnpkg for test. onert works till the below with latest master and this PR nnpkg.

$ BACKENDS="cpu;trix" ./Product/out/bin/nnpackage_run nnpackage/examples/v1.3.0/two_tflites/
Package Filename nnpackage/examples/v1.3.0/two_tflites/
0:0:0 -> 1:0:0
Error during model prepare : multiple models are not supported yet.

Jul 06 '22 04:07 glistening

How I made this example nnpkg

We can add README.md file about this.

Jul 07 '22 01:07 hseok-oh

@hseok-oh I added README.md and updated examples/README.md.

Jul 07 '22 03:07 glistening

@hseok-oh I added golden data in additional commit.

How I made golden data

$ tools/nnpackage_tool/sth2nnpkgtc/tflite2nnpkgtc.sh mv1.0_1.tflite
$ tree mv1.0_1
mv1.0_1
├── metadata
│   ├── MANIFEST
│   └── tc
│       ├── expected.h5
│       └── input.h5
└── mv1.0_1.tflite

$  cp -r mv1.0_1/metadata/tc/ nnpackage/examples/v1.3.0/two_tflites/metadata/

What operators each model has?

In short, 0 = conv2d, 1 = dconv2d

$ python tools/tflitefile_tool/model_parser.py mv1.0.tflite
[#0  (MAIN)]

#0  (MAIN) input tensors: [3]
    %3    : buffer (Empty)                   | FLOAT32 | Memory 588.0K | Shape [1, 224, 224, 3] (b'input')
#0  (MAIN) output tensors: [2]
    %2    : buffer (Empty)                   | FLOAT32 | Memory 1.5M   | Shape [1, 112, 112, 32] (b'MobilenetV1/MobilenetV1/Conv2d_0/Relu6')

#0 CONV_2D
  Fused Activation: RELU6
  Input Tensors[%3, %0, %1]
    %3    : buffer (Empty)                   | FLOAT32 | Memory 588.0K | Shape [1, 224, 224, 3] (b'input')
    %0    : buffer (&2   : [[[[ ... ]]]])    | FLOAT32 | Memory 3.4K   | Shape [32, 3, 3, 3] (b'MobilenetV1/Conv2d_0/weights')
    %1    : buffer (&1   : [ ... ])          | FLOAT32 | Memory 128.0B | Shape [32] (b'MobilenetV1/MobilenetV1/Conv2d_0/Conv2D_bias')
  Output Tensors[%2]
    %2    : buffer (Empty)                   | FLOAT32 | Memory 1.5M   | Shape [1, 112, 112, 32] (b'MobilenetV1/MobilenetV1/Conv2d_0/Relu6')
  Options
    Stride W:H = 2:2, Dilation W:H = 1:1, Padding = SAME

Number of all operator types          :    1
  CONV_2D                             :    1
Number of all operators               :    1

$ python tools/tflitefile_tool/model_parser.py mv1.1.tflite
[#0  (MAIN)]

#0  (MAIN) input tensors: [1]
    %1    : buffer (Empty)                   | FLOAT32 | Memory 1.5M   | Shape [1, 112, 112, 32] (b'MobilenetV1/MobilenetV1/Conv2d_0/Relu6')
#0  (MAIN) output tensors: [2]
    %2    : buffer (Empty)                   | FLOAT32 | Memory 1.5M   | Shape [1, 112, 112, 32] (b'MobilenetV1/MobilenetV1/Conv2d_1_depthwise/Relu6')

#0 DEPTHWISE_CONV_2D
  Fused Activation: RELU6
  Input Tensors[%1, %0, %3]
    %1    : buffer (Empty)                   | FLOAT32 | Memory 1.5M   | Shape [1, 112, 112, 32] (b'MobilenetV1/MobilenetV1/Conv2d_0/Relu6')
    %0    : buffer (&2   : [[[[ ... ]]]])    | FLOAT32 | Memory 1.1K   | Shape [1, 3, 3, 32] (b'MobilenetV1/Conv2d_1_depthwise/depthwise_weights')
    %3    : buffer (&4   : [ ... ])          | FLOAT32 | Memory 128.0B | Shape [32] (b'MobilenetV1/MobilenetV1/Conv2d_1_depthwise/depthwise_bias')
  Output Tensors[%2]
    %2    : buffer (Empty)                   | FLOAT32 | Memory 1.5M   | Shape [1, 112, 112, 32] (b'MobilenetV1/MobilenetV1/Conv2d_1_depthwise/Relu6')
  Options
    Stride W:H = 1:1, Dilation W:H = 1:1, Padding = SAME, DepthMultiplier = 1

Number of all operator types          :    1
  DEPTHWISE_CONV_2D                   :    1
Number of all operators               :    1

$ python tools/tflitefile_tool/model_parser.py mv1.0_1.tflite
[#0  (MAIN)]

#0  (MAIN) input tensors: [6]
    %6    : buffer (Empty)                   | FLOAT32 | Memory 588.0K | Shape [1, 224, 224, 3] (b'input')
#0  (MAIN) output tensors: [4]
    %4    : buffer (Empty)                   | FLOAT32 | Memory 1.5M   | Shape [1, 112, 112, 32] (b'MobilenetV1/MobilenetV1/Conv2d_1_depthwise/Relu6')

#0 CONV_2D
  Fused Activation: RELU6
  Input Tensors[%6, %0, %2]
    %6    : buffer (Empty)                   | FLOAT32 | Memory 588.0K | Shape [1, 224, 224, 3] (b'input')
    %0    : buffer (&2   : [[[[ ... ]]]])    | FLOAT32 | Memory 3.4K   | Shape [32, 3, 3, 3] (b'MobilenetV1/Conv2d_0/weights')
    %2    : buffer (&1   : [ ... ])          | FLOAT32 | Memory 128.0B | Shape [32] (b'MobilenetV1/MobilenetV1/Conv2d_0/Conv2D_bias')
  Output Tensors[%3]
    %3    : buffer (Empty)                   | FLOAT32 | Memory 1.5M   | Shape [1, 112, 112, 32] (b'MobilenetV1/MobilenetV1/Conv2d_0/Relu6')
  Options
    Stride W:H = 2:2, Dilation W:H = 1:1, Padding = SAME

#1 DEPTHWISE_CONV_2D
  Fused Activation: RELU6
  Input Tensors[%3, %1, %5]
    %3    : buffer (Empty)                   | FLOAT32 | Memory 1.5M   | Shape [1, 112, 112, 32] (b'MobilenetV1/MobilenetV1/Conv2d_0/Relu6')
    %1    : buffer (&4   : [[[[ ... ]]]])    | FLOAT32 | Memory 1.1K   | Shape [1, 3, 3, 32] (b'MobilenetV1/Conv2d_1_depthwise/depthwise_weights')
    %5    : buffer (&6   : [ ... ])          | FLOAT32 | Memory 128.0B | Shape [32] (b'MobilenetV1/MobilenetV1/Conv2d_1_depthwise/depthwise_bias')
  Output Tensors[%4]
    %4    : buffer (Empty)                   | FLOAT32 | Memory 1.5M   | Shape [1, 112, 112, 32] (b'MobilenetV1/MobilenetV1/Conv2d_1_depthwise/Relu6')
  Options
    Stride W:H = 1:1, Dilation W:H = 1:1, Padding = SAME, DepthMultiplier = 1

Number of all operator types          :    2
  CONV_2D                             :    1
  DEPTHWISE_CONV_2D                   :    1
Number of all operators               :    2

Aug 23 '22 02:08 glistening

This example seems to be verified by #9616.

@hseok-oh, @chunseoklee PTAL

Aug 29 '22 08:08 glistening

ONE ONE copied to clipboard

[nnpackage] Add multiple model nnpkg example

How I made this example nnpkg

How I made golden data

What operators each model has?

ONE
ONE copied to clipboard