InferQuantizedStreamingFCLayer error

[rubenkop]

I'm trying to implement Tincy-YOLO with Brevitas and FINN. When I get to streamline and related transformations, I get the following error with InferQuantizedStreamingFCLayer:

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-7-c0ac61a29cf1> in <module>
      7 
      8 model = model.transform(to_hls.InferBinaryStreamingFCLayer(mem_mode))
----> 9 model = model.transform(to_hls.InferQuantizedStreamingFCLayer(mem_mode))
     10 
     11 # TopK to LabelSelect

/workspace/finn-base/src/finn/core/modelwrapper.py in transform(self, transformation, make_deepcopy, cleanup, fix_float64)
    137         model_was_changed = True
    138         while model_was_changed:
--> 139             (transformed_model, model_was_changed) = transformation.apply(
    140                 transformed_model
    141             )

/workspace/finn/src/finn/transformation/fpgadataflow/convert_to_hls_layers.py in apply(self, model)
    648                             scale == 1.0 or bipolar_ok
    649                         ), "out_scale = 1.0 or bipolar output needed for conversion."
--> 650                         assert (not odt.signed()) or (
    651                             actval < 0
    652                         ), "Signed output requres actval < 0"

AssertionError: Signed output requres actval < 0

I think that the error origin might be Thresholding layers don't get "out_bias" and "out_scale" attributes when exported with Brevitas from QuantReLU activation layers. If you look at the code of the transformation, those attributes are expected when the consumer node of MatMul is a Thresholding node:

                        scale = getCustomOp(consumer).get_nodeattr("out_scale")
                        actval = getCustomOp(consumer).get_nodeattr("out_bias")
                        assert (
                            int(actval) == actval
                        ), "out_bias must be integer for HLS conversion."
                        actval = int(actval)
                        odt_is_bipolar = odt == DataType.BIPOLAR
                        bipolar_ok = (
                            odt_is_bipolar and (scale == 2.0) and (actval == -1)
                        )
                        assert (
                            scale == 1.0 or bipolar_ok
                        ), "out_scale = 1.0 or bipolar output needed for conversion."
                        assert (not odt.signed()) or (
                            actval < 0
                        ), "Signed output requres actval < 0"

Anyway, Yaman suggested that this shouldn't be a problem. So here I leave my ONNX model and the notebook that I'm using to work with it. I must mention that I'm always working on dev branches both with Brevitas and FINN. If you need any further detail I'll be happy to help.

Thank you so much in advance.

[heborras]

Hi @rubenkop, Thank you for reporting the issue! And also thank you a lot for including the ONNX model and notebook. Unfortunately, It took me quite a while to get around to take a deeper look at it. As far as I can tell there are two things happening here:

• There is a transformation missing at the beginning of the streamlining block, which is the absorb.AbsorbSignBiasIntoMultiThreshold() transformation. This transformation would normally set the out_bias attributes accordingly.
• However, the Tincy-YOLO ONNX, which was included in your issue seems to be missing some adder nodes, which the absorb.AbsorbSignBiasIntoMultiThreshold() transformation normally expects after the MultiThreshold nodes.

I'm not entirely sure why they are missing in this case, could you maybe share your network definition, which you used in Brevitas? That might give some insights.

[auphelia]

I am closing this issue due to inactivity. Please feel free to reopen or create a new issue, if you have more questions!