nncf icon indicating copy to clipboard operation
nncf copied to clipboard

Fix quantizer propagation algo

Open asenina opened this issue 3 years ago • 5 comments

Changes

Now before letting the quantizer pass through a branching node, we check that other branches coming out of the node should also be quantized. Otherwise the quantizer should not be moved.

Reason for changes

This PR fix bug(see tickets) and aligns the placement of quantizers in POT and NNCF.

image

Related tickets

#60615

Tests

Tests were added in test_quantizer_propagation_solver.py

asenina avatar Jan 17 '22 08:01 asenina

Jenkins please retry a build

asenina avatar Jan 19 '22 01:01 asenina

Jenkins please retry a build

ljaljushkin avatar Feb 04 '22 11:02 ljaljushkin

Jenkins please retry a build

asenina avatar Mar 10 '22 12:03 asenina

test this please

wonjuleee avatar Jul 05 '22 05:07 wonjuleee

Can one of the admins verify this patch?

openvino-nncf-ci avatar Oct 10 '22 17:10 openvino-nncf-ci

The newly added test_intermediate_output before and after the fix:

Before: before

After: after

Note that the intermediate output is no longer quantized since the default behaviour is to have "quantize_outputs": False, while still having activations of both convs quantized.

vshampor avatar Jan 25 '23 13:01 vshampor

Hangs on test_quantize_network[w_sym_t_a_sym_t-mask_rcnn], probably due to excessive complexity of the algo implementation. Will try to optimize.

vshampor avatar Jan 26 '23 14:01 vshampor

TF's mask-rcnn quantization keeps hanging. The issue is not with the quantization algorithm - hang occurs in applying the transformation layout to the model.

The transformation layout before and after the PR has the same amount of quantizers, but some of these have changed positions so as not to quantize outputs (since "quantize_outputs" is false by default). Here's the diff of the transformation_layout.transformations list (left is before, right is after the PR):

259,263c259,263
< 0 TargetType.AFTER_LAYER rpn-box 3 0
< 0 TargetType.AFTER_LAYER rpn-box 4 0
< 0 TargetType.AFTER_LAYER rpn-box 0 0
< 0 TargetType.AFTER_LAYER rpn-box 1 0
< 0 TargetType.AFTER_LAYER rpn-box 2 0
---
> 0 TargetType.BEFORE_LAYER tf.reshape_7 0 0
> 0 TargetType.BEFORE_LAYER tf.reshape_9 0 0
> 0 TargetType.BEFORE_LAYER tf.reshape_1 0 0
> 0 TargetType.BEFORE_LAYER tf.reshape_3 0 0
> 0 TargetType.BEFORE_LAYER tf.reshape_5 0 0
310c310
< 0 TargetType.AFTER_LAYER box-predict 0 0
---
> 0 TargetType.BEFORE_LAYER tf.reshape_35 0 0

vshampor avatar Feb 07 '23 18:02 vshampor

Launched following builds: E2E PT build: 359 - no significant changes, UNet INT8 failure known E2E TF build: 366 - no significant changes, failures in the OV column are CI-related E2E ONNX build: 356 - no visible degradations "conformance" manual build: 58 - no visible degradations

vshampor avatar Jun 12 '23 12:06 vshampor