nhc icon indicating copy to clipboard operation
nhc copied to clipboard

Speed up nhc_hw_gather_data function for KNL

Open NateCrawford opened this issue 8 years ago • 2 comments

Reduces number of access to /proc/cpuinfo in the nhc_hw_gather_data function. For processors like Intel Phi KNL with 256+ threads, the function was taking over 40 seconds to return. Running on a normal file, the function should only take about 4 seconds. Pre-filtering with grep shortens the function runtime further to 1.5s on a 1.4GHz (64 core, 256 thread) KNL.

NateCrawford avatar Sep 21 '17 21:09 NateCrawford

This one is great! On a 4 socket intel system I get the following speedup:

# stock
$ time nhc

real	0m13,799s
user	0m2,123s
sys	0m11,618s

# patched
$ time nhc

real	0m2,098s
user	0m1,956s
sys	0m0,134s

Obihoernchen avatar Jun 23 '20 18:06 Obihoernchen

Hi Why this PR is pending for so long? Is there any particular issue we should be aware of before mergin this PR into a release?

strus38 avatar Jun 30 '21 15:06 strus38

Fixed via #121 instead; closing. NHC goes to great lengths to avoid spawning subshells or other processes wherever and whenever possible. The culprit turned out to be the way the kernel and Bash handle /proc filesystem reads, so in lieu of grep, a more efficient, one-shot read-and-store method was chosen instead.

mej avatar Jun 06 '23 01:06 mej