openvino
openvino copied to clipboard
[CPU] Reduce overhead of moving to a numa node
Details:
- By combining move and execute into single function, so isExecutable() check is performed only once
- It turned out isExecutable() check is not that lightweight. For some models it adds noticeable (about 5%+) latency overhead. So, the solution is to group 'execute' and 'toNumaNode' into a single function and perform the check only once.
- fixes overhead introduced by another fix: https://github.com/openvinotoolkit/openvino/pull/23849
Tickets:
- 138220
@wangleis @sunxiaoxia2022 FYI @dmitry-gorokhov Could you please take a look?
@maxnick Please take a look