openvino icon indicating copy to clipboard operation
openvino copied to clipboard

[CPU] Reduce overhead of moving to a numa node

Open EgorDuplensky opened this issue 1 year ago • 2 comments

Details:

  • By combining move and execute into single function, so isExecutable() check is performed only once
  • It turned out isExecutable() check is not that lightweight. For some models it adds noticeable (about 5%+) latency overhead. So, the solution is to group 'execute' and 'toNumaNode' into a single function and perform the check only once.
  • fixes overhead introduced by another fix: https://github.com/openvinotoolkit/openvino/pull/23849

Tickets:

  • 138220

EgorDuplensky avatar Apr 15 '24 16:04 EgorDuplensky

@wangleis @sunxiaoxia2022 FYI @dmitry-gorokhov Could you please take a look?

EgorDuplensky avatar Apr 15 '24 16:04 EgorDuplensky

@maxnick Please take a look

dmitry-gorokhov avatar Apr 24 '24 09:04 dmitry-gorokhov