Other "compass"-like libraries

[tdd11235813]

There are other libraries around which try to provide a generic hardware discovering. What is done there and what can be re-used in compass? The ideal case would be to gather the authors and merge the work into compass. I will try to list them in this issue...

[tdd11235813]

DASH

DASH DASH, the C++ Template Library for Distributed Data Structures with Support for Hierarchical Locality for HPC and Data-Driven Science http://www.dash-project.org/ Readme

Choosing a DASH runtime (DART)
DASH provides the following variants:
    MPI: the Message Passing Interface
    CUDA: nNvidia’s Compute Unified Device Architecture (contributor distribution only)
    SHMEM: Symmetric Hierarchical Memory access (contributor distribution only)

DASH already implements a hardware information API with hierarchical levels (nested "domains").

• illustrating example on domains
• domain: can be NUMA host domain, MIC domain, unit of MIC cores, ...
  • scopes: GLOBAL > GROUP > NETWORK > NODE > MODULE > NUMA > UNIT > PACKAGE > UNCORE > CORE > CPU
• properties: cache_line_sizes, cache_shared, cache_sizes, cpu_id, max_cpu_mhz, max_threads, min_cpu_mhz, min_threads, num_cores, num_numa, num_sockets, numa_id
• license: looks like 3-clause BSD

Backend

• utilizes likwid.h, papi.h, numa.h, hwloc.h
  • see code here

Conclusion

• covers performance-relevant properties like caches and affinity => should be part of compass
• concept of domain and scope provides hierarchical links => should be part of compass as well
• likwid, papi, numa and hwloc required to gather hardware information => probably would be the same in compass
• cannot find GPU properties at the moment
• it seems DASH/DART provides a good code base for compass

[psteinb]

that is an effort highly appreciated! thanks a mllion.

[tdd11235813]

talked with a fellow from our institute, he also wants a hardware info tool, but focus is quite I/O low-level. nevertheless, there are common features and we should talk together, maybe at a GCOE meeting. but first, we should gather more information to find must-haves, nice-to-have and must-not-haves :)

[tdd11235813]

hwloc

hwloc (Open-MPI)

• source

... provides a portable abstraction (across OS, versions, architectures, ...) of the hierarchical topology of modern architectures,

• information on: NUMA, sockets, caches, processor packages & cores & units, affinities, I/O devices, hierarchy, accelerators
• C-API, highly portable
• License: 3-clause BSD

Supported OS

 hwloc supports the following operating systems:
    Linux (including old kernels not having sysfs topology information, with knowledge of cgroups, offline CPUs, ScaleMP vSMP, and NumaScale NumaConnect) on all supported hardware, including Intel Xeon Phi.
    Solaris, AIX, HP-UX and OSF/1 (a.k.a., Tru64)
    NetBSD, FreeBSD and kFreeBSD/GNU
    Darwin / OS X
    Microsoft Windows (either using MinGW or a native Visual Studio solution)
    IBM BlueGene/Q Compute Node Kernel (CNK)

Conclusion

• most likely we have to rely on it and most parts of compass just might C++-wrap hwloc

[psteinb]

yes and no, is hwloc working on Windows?

[tdd11235813]

yes it does, see updated hwloc post. We still have to check features which are not covered by hwloc et. al.

[tdd11235813]

Boost::Predef

Boost::Predef contains system information which can be retrieved by the compiler.

provides information about:

• architecture, compiler, language, OS, SIMD type, endianess
• probably not important: [C] library, platform (mingw, windows [desktop, phone, runtime, store])

First released in Boost 1.55.0.

Conclusion

• could replace our compile-time stuff, which would introduce dependency of Boost >=1.55.0

[psteinb]

and of course we should not forget cpu_features