oVo
oVo
@l-bat @MaximProshin > have you measured accuracy of initial (FP32/FP16) OpenVINO IR? | model | coco_precision | Throughput | --- | --- | --- | | FP32 | 31.93% |...
@l-bat Performance degradation is also observed in 240cc24. Throughput degradation is minor when I retried. | Model | coco_precision | Throughput | | --- | --- | --- | |...
Note that I hope nncf (native-backend) will yield the same accuracy/throughput with pot-backend because `use_pot=True` will be deleted in the future (https://github.com/openvinotoolkit/nncf/issues/1923#issuecomment-1598277970)
Oh, I misread that. 31.33% was the int8(native) accuracy. How was the accuracy of FP32 model? BTW, I'll try nncf/tests/openvino/tools/calibrate.py tomorrow.
@l-bat > Why do you use has_background: False parameter in accuracy_check.yaml? Regarding dataset settings, I copied the following sample - https://github.com/openvinotoolkit/open_model_zoo/blob/c7a13a842d41333397ea6c3f9bc5a7053da00eec/models/public/efficientdet-d0-tf/accuracy-check.yml#L7C15-L7C60 - https://github.com/openvinotoolkit/open_model_zoo/blob/c7a13a842d41333397ea6c3f9bc5a7053da00eec/data/dataset_definitions.yml#L119 > Could you please try to use...
@l-bat Sorry for the confusion. It should have been named `ms_coco_detection_90_class_without_background`. While looking for a working config, the name seems to have been inappropriate. I didn't use the --definitions option...
@l-bat From the above, the following problems were identified at the latest NNCF - native+preset=MIXED: the accuracy and throughput is worse than pot/nncf240 - native+preset=PERFORMANCE: accuracy is good, but throughput...
@l-bat sorry for the late reply. I tested commit 76d6b3a. | Model | coco_precision | `benchmark-app -api sync` | | --- | --- | --- | | FP32 | 31.93%...