proof-of-concept for cpu topology tree

[mrvn]

This is just a quick proof-of-concept for getting the topology of the cpus in a system in tree form. Its only to illustrate my comment in issue #550 "Enhancement: get threads per core" and only for Linux, not to be actually merged yet.

[SimonHeimberg]

On my linux machine, this would work perfect.

Not sure if it would work on every system. I found some formats which would not: https://unix.stackexchange.com/questions/255558/understanding-the-output-of-proc-cpuinfo (AArch64, in 1 section, 7 years old) https://github.com/pytorch/cpuinfo/blob/main/test/cpuinfo/xiaomi-redmi-note-3.log (Aarch64, in 1 section) https://github.com/pytorch/cpuinfo/blob/main/test/cpuinfo/arndaleboard.log has additional section https://github.com/pytorch/cpuinfo/blob/main/test/cpuinfo/raspberrypi3.log has additional section https://github.com/pytorch/cpuinfo/blob/main/test/cpuinfo/atm7029b-tablet.log (ARMv7) has additional section

All examples would work when a new section would be started on ^processor:[ \t]*[0-9], but some values would have to be adapted.

https://github.com/pytorch/cpuinfo/blob/main/test/cpuinfo/raspberrypi.log does not even have "processor" :see_no_evil:

[SimonHeimberg]

More concerning linux:

Looks like /proc/cpuinfo is meant for human consumption and thus lacks a stable format (from https://stackoverflow.com/a/23378780). Makes it hard to use. lscpu -p gives the topology information (and allows to select columns with -p=col1,col2), but is not available on every system. (from same so question)

``lscpu -p also shows the tree structure for linux: CPU,Core,Socket,Node (Node is missing in your code. No idea yet what a node is.)

Some inspiration might come from lscpu from linux-utils. It uses information from /sys/devices/system/cpu/cpu[0-9]*/topology/*_siblings and from /proc/cpu_info. Their advantage is that the development is together with the kernel version, so they know what virtual files are around. Actually count_cpu_cores does a similar thing already, it first tries /sys/... and falls back to /proc/... . The first one should be more reliable here as well.

Other os: seems to be difficult on FreeBsd: https://stackoverflow.com/questions/4083848/what-is-the-equivalent-of-proc-cpuinfo-on-freebsd-v8-1 windows maybe https://learn.microsoft.com/en-us/windows/win32/api/sysinfoapi/nf-sysinfoapi-getlogicalprocessorinformation

[SimonHeimberg]

Wondering if another structure would be more useful for many cases. Functions like os.sched_getaffinity() or psutil.Process().cpu_affinity() work with the id of the logical cpu. Taking the (logical) cpu id as base, we get something like

(LogicalCpu(cpu_id=0, core_id=0, socket_id=0, node_id=0, l1d_id=0, l1i_id=0, l2_id=0, l3_id=0, online='Y'),
 LogicalCpu(cpu_id=1, core_id=0, socket_id=0, node_id=0, l1d_id=0, l1i_id=0, l2_id=0, l3_id=0, online='Y'),
 LogicalCpu(cpu_id=2, core_id=1, socket_id=0, node_id=0, l1d_id=1, l1i_id=1, l2_id=1, l3_id=0, online='Y'),
...

with LogicalCpu = collections.namedtuple("LogicalCpu", ("cpu_id", "core_id", ...) (or a class)

The tree structure makes sense for other information (vendor, model name, flags), and for understanding how the systems "cpu" structure looks like (it took me a long time).