finn icon indicating copy to clipboard operation
finn copied to clipboard

Extend support for the resize operator

Open klassen9 opened this issue 1 year ago • 1 comments

The PR extends/fixes the current implementations for the transformations of the resize operator in FINN.

We wanted to streamline a model that uses a resize operator, but the current implementation only supports resize via the scales input. The model we used, uses the sizes input. scales and sizes are both responsible for determining the amount of resizing and can be translated into each other. They are also mutually exclusive.

A problem that arose while implementing the support for sizes was to determine which input is used. Different versions of the Onnx opset have a different understanding of how to mark an input as unused. As an example:

  • Version 11 dictates: "If sizes is needed, the user must set scales to an empty tensor."
  • Version 13 dictates: "If sizes is needed, the user can use an empty string as the name of scales in this operator’s input list."

I am thus checking if scales or sizes exist by checking if they are None or are empty to support all versions. None is returned by model.get_initializer() if the input is an empty string in the input list.

Furthermore, this PR adds support for the Version 10 resize operation. The Version 10 resize operation only supports the scales input.

Other than the scales and sizes inputs and the input tensor, there is the roi (region of interest) input. The roi is not supported by the hardware implementation and therefore triggers an assert in convert_to_hw_layers.py when used.

The test_scale_resize_nhwc.py test was extended by a test that uses the sizes input instead of the scales input.

I also published a PR for qonnx adding the resize operation to the change_3d_tensors_to_4d transformation (https://github.com/fastmachinelearning/qonnx/pull/125).

klassen9 avatar Jul 06 '24 17:07 klassen9

Sorry, I didn't apply pre-commit correctly. Should be fixed in the second commit.

klassen9 avatar Jul 07 '24 10:07 klassen9