openvino icon indicating copy to clipboard operation
openvino copied to clipboard

Fix performance regression of conformance tests

Open sunxiaoxia2022 opened this issue 1 year ago • 4 comments

Details:

  1. fix performance regression of conformance tests

    Root cause is make_default_multi_threaded() API which was refactored in PR22414. This API is used to calculate the number of threads. Master version: When the test machine has both Pcore and Ecore, it recognized the machine type and will select proper cores (Pcores are selected in general machine which pcores >Ecores / 2) to create executor. Old version: the input parameter _threadBindingType of config was used to judge the type of current machine, and _threadBindingType is always set to default value None in template plugin. So hybrid core machine is regarded to core machine and physical cores (include Pcores and Ecores) are used. _threadBindingType is deprecated in master now. For example: ADL i9-12900K, 8 Pcore, 8Ecore. Old version: threads=16 in any cores. Master: threads=8 in Pcores. Actually, the threads calculated from make_default_multi_threaded of master version is the best option in normal situation which not create multi threads in app side. But in conformance test, 24 threads are created to seize CPU resource. Of cause, all cores are must be used to achieved the best performance.

  2. fix compile model time slow down when user set streams to AUTO

Tickets:

  • 131820

sunxiaoxia2022 avatar Feb 20 '24 02:02 sunxiaoxia2022

should we also revert https://github.com/openvinotoolkit/openvino/pull/22735 ?

ilya-lavrenov avatar Feb 20 '24 08:02 ilya-lavrenov

@sunxiaoxia2022 Please provide details in description regarding root-cause of the issue and how exactly this PR helps to fix it

dmitry-gorokhov avatar Feb 20 '24 08:02 dmitry-gorokhov

@sunxiaoxia2022 Please provide details in description regarding root-cause of the issue and how exactly this PR helps to fix it

@dmitry-gorokhov updated in details, please take a look, thank you.

sunxiaoxia2022 avatar Feb 23 '24 01:02 sunxiaoxia2022

should we also revert #22735 ?

Yes, we can revert it after this PR.

sunxiaoxia2022 avatar Feb 23 '24 01:02 sunxiaoxia2022

@sunxiaoxia2022 Please add revert of https://github.com/openvinotoolkit/openvino/pull/22735 into this PR

dmitry-gorokhov avatar Feb 26 '24 07:02 dmitry-gorokhov