openvino
openvino copied to clipboard
[WIP] [CPU][Convert] Add temporary change to conformace tests to fix Convert problem for FP16->integral type
[CPU][Convert] Add temporary change to conformance tests to fix Convert problem for FP16->integral type
Details:
- This is a temporary PR to show an example how to fix Convert problem for FP16->integral type in conformance tests
Ticket: 89296
@dmitry-gorokhov Could anyone have a look at this PR? It is a possible fix for - ./openvino/bin/intel64/Debug/conformanceTests --input_folders ./ov-master/conformance_omz/_backup/Convert/i64 --gtest_filter=Convert_1178515.xml The original ticket is unknown.
I don't think this is the right place to adjust threshold for particular test. @iefode may I ask you to recommend proper solution here? @apavliuk-altran could you please also provide more details why the threshold should be adjusted and how value = 1.0001 was chosen?
I don't think this is the right place to adjust threshold for particular test. @iefode may I ask you to recommend proper solution here? @apavliuk-altran could you please also provide more details why the threshold should be adjusted and how value = 1.0001 was chosen?
@dmitry-gorokhov, the problem with this conformance test arises because during a Convert test with fp16 input precision and int64 output precision, some additional conversions are added, one of which is fp32->fp16 conversion and during this conversion some fp32 values, e.g. 9.99609375 are converted to fp16 not as 9.99... but as 10.0, and subsequently are converted to int64 as 10.
So, one of the ways to solve this issue is to increase the threshold of the test to a value greater than 1.0, e.g. 1.0001.
adding @lohika-denis-kotov
@iefode
This way is not applicable to fix the issue with Convert test. The threshold is so much for single operation graph. Moreover, it's very rude workaround
This PR was created to start discussion about how to do it properly. For single layer tests we have possibility to modify threshold by creating our own class that inherited from original single layer test. But in for conformance tests it is not possible, because we do not have a separate plugin conformance test and cannot modify threshold locally for particular plugin/test_case
@apavliuk-altran i am going to close the pr. Do you have any concerns?