Yggdrasil [WIP] Add likwid 5.2.1

[WIP] Add likwid 5.2.1

Open carstenbauer opened this issue 2 years ago • 23 comments

This is my attempt to provide a basic configuration of LIKWID, in particular the library liblikwid, via BinaryBuilder (https://github.com/JuliaPerf/LIKWID.jl/issues/24). The plan is to later use this as a fallback in LIKWID.jl (if there is no system-wide likwid available).

TODOs:

[x] Reuse hwloc and Lua JLLs
[x] use a patch instead of sed
[x] fix ldd: --version issues (as much as possible)

Potential TODOs:

[ ] provide the executable lua scripts (e.g. likwid-topology, likwid-perfctr, etc.) as FileProducts -> difficult due to absolute paths
[ ] support for musl (if possible)
[ ] support for aarch64 and powerpc?

Questions / Remarks:

To modify the config.mk, I've opted for three sed commands. (Should I use a patch instead? What would be the advantage / disadvantage?)
I get the following make log:

sandbox:${WORKSPACE}/srcdir/likwid-5.2.1 # make
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
===>  GENERATE HEADER GCC/perfmon_a15_events.h
===>  GENERATE HEADER GCC/perfmon_a57_events.h
===>  GENERATE HEADER GCC/perfmon_a64fx_events.h
[ ... many more GENERATE HEADERs ... ]
===>  GENERATE HEADER GCC/perfmon_zen2_events.h
===>  GENERATE HEADER GCC/perfmon_zen3_events.h
===>  GENERATE HEADER GCC/perfmon_zen_events.h
===>  COMPILE  GCC/access.o
===>  COMPILE  GCC/access_client.o
===>  COMPILE  GCC/access_x86.o
[ ... many more COMPILEs ...]
===>  COMPILE  GCC/tree.o
===>  COMPILE  GCC/voltage.o
===>  COMPILE  GCC/loadData.o
===>  ENTER  /workspace/srcdir/likwid-5.2.1/ext/hwloc
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
===>  ENTER  /workspace/srcdir/likwid-5.2.1/ext/lua
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
===>  CREATE SHARED LIB  liblikwid.so
===>  CREATE LIB  liblikwidpin.so
make[1]: Entering directory '/workspace/srcdir/likwid-5.2.1/src/pthread-overload'
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
make[1]: Leaving directory '/workspace/srcdir/likwid-5.2.1/src/pthread-overload'
===>  ADJUSTING  likwid-perfctr
===>  ADJUSTING  likwid-pin
===>  ADJUSTING  likwid-powermeter
===>  ADJUSTING  likwid-topology
===>  ADJUSTING  likwid-memsweeper
===>  ADJUSTING  likwid-mpirun
===>  ADJUSTING  likwid-features
===>  ADJUSTING  likwid-perfscope
===>  ADJUSTING  likwid-genTopoCfg
===>  ADJUSTING  likwid.lua
===>  ENTER  bench
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
===>  COMPILE C GCC/allocator.o
===>  COMPILE C GCC/barrier.o
===>  COMPILE C GCC/bench.o
===>  COMPILE C GCC/bstrlib.o
===>  COMPILE C GCC/bstrlib_helper.o
===>  COMPILE C GCC/ptt2asm.o
===>  COMPILE C GCC/strUtil.o
===>  COMPILE C GCC/threads.o
===>  GENERATE BENCHMARKS
===>  ASSEMBLE  GCC/clcopy.o
===>  ASSEMBLE  GCC/clload.o
[ ... many more ASSEMBLEs ...]
===>  ASSEMBLE  GCC/update_sp_sse.o
===>  ASSEMBLE  GCC/update_sse.o
===>  LINKING  likwid-bench

Note the

ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled

bits. Should I care about those given that the build process seems to proceed just fine?

(@TomTheBear, @Suavesito-Olimpiada FYI)

May 14 '22 21:05 carstenbauer

Patch tends to be more stable since it will fail if it couldn't apply it.

Ideally you would be reusing HWLoc_jll instead of building it again, same with lua

May 14 '22 23:05 vchuravy

@carstenbauer: The ldd --version call is used to determine the glibc version because old glibc version lack the wrappers for sched_setaffinity. You can just add -DHAS_SCHEDAFFINITY to the DEFINES to forcefully use the wrapper (which should be available on all systems nowadays).

I never tried building LIKWID with musl.

For building with external hwloc and lua, check spack's LIKWID package.

May 15 '22 00:05 TomTheBear

Ideally you would be reusing HWLoc_jll instead of building it again, same with lua

Yes, indeed, just forgot to put it on the Todo list.

May 15 '22 05:05 carstenbauer

You can just add -DHAS_SCHEDAFFINITY to the DEFINES to forcefully use the wrapper (which should be available on all systems nowadays).

@TomTheBear Can you elaborate a bit more, i.e. where should I add it exactly? I tried setting it in make/config_defines.mk (and make/include_GCC.mk / make/include_GCCX86.mk) without success.

May 15 '22 10:05 carstenbauer

Can you elaborate a bit more, i.e. where should I add it exactly? I tried setting it in make/config_defines.mk (and make/include_GCC.mk / make/include_GCCX86.mk) without success.

If you add it to the DEFINES in make/include_<COMPILER>.mk, it is present on the compile lines. You can check with make Q=:

$ make Q=
===>  COMPILE  GCC/perfmon.o
gcc [...] -DHAS_SCHEDAFFINITY [...]

May 15 '22 13:05 TomTheBear

@TomTheBear That's what I did (and it works in the sense that you describe) but I still get

sandbox:${WORKSPACE}/srcdir/likwid-5.2.1 # make Q=
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
===>  GENERATE HEADER GCC/perfmon_a15_events.h
./perl/gen_events.pl src/includes/perfmon_a15_events.txt GCC/perfmon_a15_events.h
===>  GENERATE HEADER GCC/perfmon_a57_events.h
./perl/gen_events.pl src/includes/perfmon_a57_events.txt GCC/perfmon_a57_events.h
===>  GENERATE HEADER GCC/perfmon_a64fx_events.h
./perl/gen_events.pl src/includes/perfmon_a64fx_events.txt GCC/perfmon_a64fx_events.h
===>  GENERATE HEADER GCC/perfmon_atom_events.h
./perl/gen_events.pl src/includes/perfmon_atom_events.txt GCC/perfmon_atom_events.h
===>  GENERATE HEADER GCC/perfmon_broadwellEP_events.h
./perl/gen_events.pl src/includes/perfmon_broadwellEP_events.txt GCC/perfmon_broadwellEP_events.h
===>  GENERATE HEADER GCC/perfmon_broadwell_events.h
./perl/gen_events.pl src/includes/perfmon_broadwell_events.txt GCC/perfmon_broadwell_events.h
===>  GENERATE HEADER GCC/perfmon_broadwelld_events.h
./perl/gen_events.pl src/includes/perfmon_broadwelld_events.txt GCC/perfmon_broadwelld_events.h
===>  GENERATE HEADER GCC/perfmon_cascadelakeX_events.h
./perl/gen_events.pl src/includes/perfmon_cascadelakeX_events.txt GCC/perfmon_cascadelakeX_events.h
===>  GENERATE HEADER GCC/perfmon_cavtx2_events.h
./perl/gen_events.pl src/includes/perfmon_cavtx2_events.txt GCC/perfmon_cavtx2_events.h
===>  GENERATE HEADER GCC/perfmon_core2_events.h
./perl/gen_events.pl src/includes/perfmon_core2_events.txt GCC/perfmon_core2_events.h
===>  GENERATE HEADER GCC/perfmon_goldmont_events.h
./perl/gen_events.pl src/includes/perfmon_goldmont_events.txt GCC/perfmon_goldmont_events.h
===>  GENERATE HEADER GCC/perfmon_haswellEP_events.h
./perl/gen_events.pl src/includes/perfmon_haswellEP_events.txt GCC/perfmon_haswellEP_events.h
===>  GENERATE HEADER GCC/perfmon_haswell_events.h
./perl/gen_events.pl src/includes/perfmon_haswell_events.txt GCC/perfmon_haswell_events.h
===>  GENERATE HEADER GCC/perfmon_icelakeX_events.h
./perl/gen_events.pl src/includes/perfmon_icelakeX_events.txt GCC/perfmon_icelakeX_events.h
===>  GENERATE HEADER GCC/perfmon_icelake_events.h
./perl/gen_events.pl src/includes/perfmon_icelake_events.txt GCC/perfmon_icelake_events.h
===>  GENERATE HEADER GCC/perfmon_interlagos_events.h
./perl/gen_events.pl src/includes/perfmon_interlagos_events.txt GCC/perfmon_interlagos_events.h
===>  GENERATE HEADER GCC/perfmon_ivybridgeEP_events.h
./perl/gen_events.pl src/includes/perfmon_ivybridgeEP_events.txt GCC/perfmon_ivybridgeEP_events.h
===>  GENERATE HEADER GCC/perfmon_ivybridge_events.h
===>  GENERATE HEADER GCC/perfmon_k10_events.h
./perl/gen_events.pl src/includes/perfmon_k10_events.txt GCC/perfmon_k10_events.h
===>  GENERATE HEADER GCC/perfmon_k8_events.h
./perl/gen_events.pl src/includes/perfmon_k8_events.txt GCC/perfmon_k8_events.h
===>  GENERATE HEADER GCC/perfmon_kabini_events.h
./perl/gen_events.pl src/includes/perfmon_kabini_events.txt GCC/perfmon_kabini_events.h
===>  GENERATE HEADER GCC/perfmon_knl_events.h
./perl/gen_events.pl src/includes/perfmon_knl_events.txt GCC/perfmon_knl_events.h
===>  GENERATE HEADER GCC/perfmon_nehalemEX_events.h
./perl/gen_events.pl src/includes/perfmon_nehalemEX_events.txt GCC/perfmon_nehalemEX_events.h
===>  GENERATE HEADER GCC/perfmon_nehalem_events.h
./perl/gen_events.pl src/includes/perfmon_nehalem_events.txt GCC/perfmon_nehalem_events.h
===>  GENERATE HEADER GCC/perfmon_neon1_events.h
./perl/gen_events.pl src/includes/perfmon_neon1_events.txt GCC/perfmon_neon1_events.h
===>  GENERATE HEADER GCC/perfmon_p6_events.h
./perl/gen_events.pl src/includes/perfmon_p6_events.txt GCC/perfmon_p6_events.h
===>  GENERATE HEADER GCC/perfmon_phi_events.h
./perl/gen_events.pl src/includes/perfmon_phi_events.txt GCC/perfmon_phi_events.h
===>  GENERATE HEADER GCC/perfmon_pm_events.h
./perl/gen_events.pl src/includes/perfmon_pm_events.txt GCC/perfmon_pm_events.h
===>  GENERATE HEADER GCC/perfmon_power8_events.h
./perl/gen_events.pl src/includes/perfmon_power8_events.txt GCC/perfmon_power8_events.h
===>  GENERATE HEADER GCC/perfmon_power9_events.h
./perl/gen_events.pl src/includes/perfmon_power9_events.txt GCC/perfmon_power9_events.h
===>  GENERATE HEADER GCC/perfmon_sandybridgeEP_events.h
./perl/gen_events.pl src/includes/perfmon_sandybridgeEP_events.txt GCC/perfmon_sandybridgeEP_events.h
===>  GENERATE HEADER GCC/perfmon_sandybridge_events.h
./perl/gen_events.pl src/includes/perfmon_sandybridge_events.txt GCC/perfmon_sandybridge_events.h
===>  GENERATE HEADER GCC/perfmon_silvermont_events.h
./perl/gen_events.pl src/includes/perfmon_silvermont_events.txt GCC/perfmon_silvermont_events.h
===>  GENERATE HEADER GCC/perfmon_skylakeX_events.h
./perl/gen_events.pl src/includes/perfmon_skylakeX_events.txt GCC/perfmon_skylakeX_events.h
===>  GENERATE HEADER GCC/perfmon_skylake_events.h
./perl/gen_events.pl src/includes/perfmon_skylake_events.txt GCC/perfmon_skylake_events.h
===>  GENERATE HEADER GCC/perfmon_tigerlake_events.h
./perl/gen_events.pl src/includes/perfmon_tigerlake_events.txt GCC/perfmon_tigerlake_events.h
===>  GENERATE HEADER GCC/perfmon_westmereEX_events.h
./perl/gen_events.pl src/includes/perfmon_westmereEX_events.txt GCC/perfmon_westmereEX_events.h
===>  GENERATE HEADER GCC/perfmon_westmere_events.h
./perl/gen_events.pl src/includes/perfmon_westmere_events.txt GCC/perfmon_westmere_events.h
===>  GENERATE HEADER GCC/perfmon_zen2_events.h
./perl/gen_events.pl src/includes/perfmon_zen2_events.txt GCC/perfmon_zen2_events.h
===>  GENERATE HEADER GCC/perfmon_zen3_events.h
./perl/gen_events.pl src/includes/perfmon_zen3_events.txt GCC/perfmon_zen3_events.h
===>  GENERATE HEADER GCC/perfmon_zen_events.h
./perl/gen_events.pl src/includes/perfmon_zen_events.txt GCC/perfmon_zen_events.h
===>  COMPILE  GCC/access.o
gcc -c  -O2 -std=c99 -Wno-format -fPIC -fPIC -fvisibility=hidden   -DPAGE_ALIGNMENT=4096 -DLIKWID_MONITOR_LOCK -DDEBUGLEV=0 -DHAS_SCHEDAFFINITY -DVERSION=5 -DRELEASE=2 -DMINORVERSION=1 -DCFGFILE=/etc/likwid.cfg -DTOPOFILE=/etc/likwid_topo.cfg -DINSTALL_PREFIX=/workspace/destdir -DMAX_NUM_THREADS=300 -DMAX_NUM_NODES=64 -DMAX_NUM_CLIARGS=16384 -DACCESSDAEMON=/workspace/destdir/sbin/likwid-accessD -DFREQDAEMON=/workspace/destdir/sbin/likwid-setFreq -DGROUPPATH=/workspace/destdir/share/likwid/perfgroups -DLIKWIDLOCK=/var/run/likwid.lock -DLIKWIDSOCKETBASE=/tmp/likwid   -DGITCOMMIT=233ab943543480cd46058b34616c174198ba0459 -D_GNU_SOURCE -DCOLOR=BLUE -DLIKWID_USE_PERFEVENT -DHAS_MEMPOLICY -DHAS_SCHEDAFFINITY -DLIKWID_USE_HWLOC -DLIKWID_USE_PERFEVENT -DACCESSMODE=-1 -I./src/includes -I/workspace/srcdir/likwid-5.2.1/ext/lua/includes -I/workspace/srcdir/likwid-5.2.1/ext/hwloc/include -I./GCC ./src/access.c -o GCC/access.o
[....]

Do I need to put it somewhere else as well?

May 15 '22 17:05 carstenbauer

The ldd --version call is used to determine the glibc version because old glibc version lack the wrappers for sched_setaffinity.

sandbox:${WORKSPACE} # echo '#include <features.h>' | cc -dM -E - | grep -E '#define __GLIBC_(|MINOR_)_'
#define __GLIBC__ 2
#define __GLIBC_MINOR__ 12

is probably more cross-compilation friendly than running ldd on a system where ldd is not from glibc.

Related to the topic of cross-compilation, we looked into building likwid some time ago, but I remember it wasn't really possible to do a cross-build because the package needed some information about the target system at compile time, or some local kernel modules, or something like that. Is this still the case?

May 15 '22 18:05 giordano

For building with external hwloc and lua, check spack's LIKWID package.

It seems like likwid still installs libhwloc.so.5.2 (and makes the libhwloc.so symlink point to it) when I specify an external hwloc. This is the $libdir before and after make install [...]:

Before

lrwxrwxrwx    1 root     root            18 May 15 17:31 libhwloc.so -> libhwloc.so.15.5.3*
lrwxrwxrwx    1 root     root            18 May 15 17:31 libhwloc.so.15 -> libhwloc.so.15.5.3*
lrwxrwxrwx    1 root     root            79 May 15 17:31 libhwloc.so.15.5.3 -> ../../artifacts/f11a525a0a63e8dac0e0236c0e578c0d8e34d59e/l
ib/libhwloc.so.15.5.3*

After

lrwxrwxrwx    1 root     root            15 May 15 17:52 libhwloc.so -> libhwloc.so.5.2*
lrwxrwxrwx    1 root     root            18 May 15 17:31 libhwloc.so.15 -> libhwloc.so.15.5.3*
lrwxrwxrwx    1 root     root            79 May 15 17:31 libhwloc.so.15.5.3 -> ../../artifacts/f11a525a0a63e8dac0e0236c0e578c0d8e34d59e/l
ib/libhwloc.so.15.5.3*
lrwxrwxrwx    1 root     root            15 May 15 17:52 libhwloc.so.5 -> libhwloc.so.5.2*
-rwxr-xr-x    1 root     root       1729301 May 15 17:52 libhwloc.so.5.2*

I see that they apply a patch in the easyconfig that addresses this. @TomTheBear Why isn't the easyconfig patch the default? I.e., is there a deeper reason for the creation of libhwloc.so.5.2? Otherwise I'd apply a similar patch.

May 15 '22 18:05 carstenbauer

The script bin/likwid-topology(and probably also the others) hardcodes local absolute paths of the build environment, which won't exist at runtime. That's not going to work.

May 15 '22 19:05 giordano

The script bin/likwid-topology(and probably also the others) hardcodes local absolute paths of the build environment, which won't exist at runtime. That's not going to work.

Yeah. That's certainly unfortunate. But the liblikwid.so is the primary reason for me to attempt to create a JLL. You can use lots (most?) of the features without the command-line tools. (One could argue that the "in-Julia" interface to LIKWID is anyways nicer / more julian than the CLI.)

May 15 '22 19:05 carstenbauer

support for musl (if possible)

It'd be nice to build for all Linux platforms, if possible (at least glibc ones, if that's a requirement). I'm thinking in particular of aarch64 and powerpc64le, which are more popular in HPC settings.

May 15 '22 19:05 giordano

Why isn't the easyconfig patch the default? I.e., is there a deeper reason for the creation of libhwloc.so.5.2?

@carstenbauer: I test LIKWID only in the default configuration varying important options (COMPILER and ACCESSMODE). The feature to use external lua and hwloc was only added for the package management folks out there. The reason is quite simple, there is not much feedback from the package management folks. I didn't even know that an easy_build recipe exists for LIKWID. And consequently, I didn't know the patch. The library is probably not needed as the easy_build guys (hopefully) tested their stuff.

May 16 '22 10:05 TomTheBear

The script bin/likwid-topology(and probably also the others) hardcodes local absolute paths of the build environment, which won't exist at runtime. That's not going to work.

@giordano This can be avoided by getting the script path in the beginning. It's just Lua. Patch it if you need it differently. Until now, Yggdrasil seems to be the only package manager having problems with absolute paths.

May 16 '22 10:05 TomTheBear

Until now, Yggdrasil seems to be the only package manager having problems with absolute paths.

This is the only sensible thing given that Yggdrasil

does not have administrator permissions on systems,
is cross-compiled on a Linux system,
want to support multiple distribution and versions of different OS's,
and must be relocatable.

The last point might be the most important of all.

May 16 '22 13:05 Suavesito-Olimpiada

This is the only sensible thing given that Yggdrasil [...]

While true, that's a Julia / Yggdrasil perspective. A tool (like likwid) may not care about what Yggdrasil wants / needs. It's a tedious task to cater to the specific needs of various package managers so we can't expect non-Julia developers to spend much time on it. We can hope though 😉

Fortunately, in this case, @TomTheBear has already committed a few simple patches to likwid so that from version 5.2.2 on

the ldd --version issue should be gone (https://github.com/RRZE-HPC/likwid/commit/d6bbdcf2a84210d9c1dd6952d1ae533ae36bff07) and
we don't need to patch the Makefile any more (i.e. we can drop the easyconfig-like patch) (https://github.com/RRZE-HPC/likwid/commit/3a79b059c57f03949a74977b654a416b5ed51fcf)

Thanks for this! ❤️

May 16 '22 16:05 carstenbauer

A tool (like likwid) may not care about what Yggdrasil wants / needs.

Absolutely, I just wanted to explain the perspective of Julia/Yggdrasil, not to rant about the projects that don't do it, or ask for every project to do it. I'm sorry if my comment read otherwise. :sweat_smile:

Fortunately, in this case, @TomTheBear has already committed a few simple patches to likwid

Thanks, @TomTheBear! That sounds incredible. :tada:

May 18 '22 16:05 Suavesito-Olimpiada

It'd be nice to build for all Linux platforms, if possible (at least glibc ones, if that's a requirement). I'm thinking in particular of aarch64 and powerpc64le, which are more popular in HPC settings.

PowerPC FWIW, if I set COMPILER = GCCPOWER in config.mk (and use external hwloc and lua) compilation works without errors. Q: How do I set COMPILER = GCCPOWER for powerpc and COMPILER = GCC for regular x86_64 (i.e. system dependent patches)?

aarch64 COMPILER = GCCARMv8 doesn't work:

sandbox:${WORKSPACE}/srcdir/likwid-5.2.1 # make PREFIX=${prefix} HWLOC_INCLUDE_DIR=${includedir} HWLOC_LIB_DIR=${libdir} HWLOC_LIB_NAME=hwloc LUA_INCLUDE_DIR=${includedir} LUA_LIB_DIR=${libdir} LUA_LIB_NAME=lua LUA_BIN=${bindir}
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
===>  GENERATE HEADER GCCARMv8/perfmon_a15_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_a57_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_a64fx_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_atom_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_broadwellEP_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_broadwell_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_broadwelld_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_cascadelakeX_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_cavtx2_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_core2_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_goldmont_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_haswellEP_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_haswell_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_icelakeX_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_icelake_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_interlagos_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_ivybridgeEP_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_ivybridge_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_k10_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_k8_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_kabini_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_knl_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_nehalemEX_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_nehalem_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_neon1_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_p6_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_phi_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_pm_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_power8_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_power9_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_sandybridgeEP_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_sandybridge_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_silvermont_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_skylakeX_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_skylake_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_tigerlake_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_westmereEX_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_westmere_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_zen2_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_zen3_events.h
===>  GENERATE HEADER GCCARMv8/perfmon_zen_events.h
===>  COMPILE  GCCARMv8/access.o
BinaryBuilder: Cannot force an architecture via -march
make: *** [Makefile:302: GCCARMv8/access.o] Error 1

i686 COMPILER = GCCX86 works (with warnings)

sandbox:${WORKSPACE}/srcdir/likwid-5.2.1 # make PREFIX=${prefix} HWLOC_INCLUDE_DIR=${includedir} HWLOC_LIB_DIR=${libdir} HWLO
C_LIB_NAME=hwloc LUA_INCLUDE_DIR=${includedir} LUA_LIB_DIR=${libdir} LUA_LIB_NAME=lua LUA_BIN=${bindir}                      
ldd: --version: No such file or directory                                                                                    
sh: 4: unknown operand                                                                                                       
Info: Compiling for perf_event interface. Measurements of thermal information is disabled                                    
===>  GENERATE HEADER GCCX86/perfmon_a15_events.h                                                                            
===>  GENERATE HEADER GCCX86/perfmon_a57_events.h                                                                            
===>  GENERATE HEADER GCCX86/perfmon_a64fx_events.h                                                                          
===>  GENERATE HEADER GCCX86/perfmon_atom_events.h                                                                           
===>  GENERATE HEADER GCCX86/perfmon_broadwellEP_events.h                                                                    
===>  GENERATE HEADER GCCX86/perfmon_broadwell_events.h                                                                      
===>  GENERATE HEADER GCCX86/perfmon_broadwelld_events.h                                                                     
===>  GENERATE HEADER GCCX86/perfmon_cascadelakeX_events.h                                                                   
===>  GENERATE HEADER GCCX86/perfmon_cavtx2_events.h                                                                         
===>  GENERATE HEADER GCCX86/perfmon_core2_events.h                                                                          
===>  GENERATE HEADER GCCX86/perfmon_goldmont_events.h                                                                       
===>  GENERATE HEADER GCCX86/perfmon_haswellEP_events.h                                                                      
===>  GENERATE HEADER GCCX86/perfmon_haswell_events.h                                                                        
===>  GENERATE HEADER GCCX86/perfmon_icelakeX_events.h                                                                       
===>  GENERATE HEADER GCCX86/perfmon_icelake_events.h                                                                        
===>  GENERATE HEADER GCCX86/perfmon_interlagos_events.h                                                                     
===>  GENERATE HEADER GCCX86/perfmon_ivybridgeEP_events.h                                                                    
===>  GENERATE HEADER GCCX86/perfmon_ivybridge_events.h                                                                      
===>  GENERATE HEADER GCCX86/perfmon_k10_events.h                                                                            
===>  GENERATE HEADER GCCX86/perfmon_k8_events.h                                                                             
===>  GENERATE HEADER GCCX86/perfmon_kabini_events.h                                                                         
===>  GENERATE HEADER GCCX86/perfmon_knl_events.h
===>  GENERATE HEADER GCCX86/perfmon_nehalemEX_events.h
===>  GENERATE HEADER GCCX86/perfmon_nehalem_events.h
===>  GENERATE HEADER GCCX86/perfmon_neon1_events.h
===>  GENERATE HEADER GCCX86/perfmon_p6_events.h
===>  GENERATE HEADER GCCX86/perfmon_phi_events.h
===>  GENERATE HEADER GCCX86/perfmon_pm_events.h
===>  GENERATE HEADER GCCX86/perfmon_power8_events.h
===>  GENERATE HEADER GCCX86/perfmon_power9_events.h
===>  GENERATE HEADER GCCX86/perfmon_sandybridgeEP_events.h
===>  GENERATE HEADER GCCX86/perfmon_sandybridge_events.h
===>  GENERATE HEADER GCCX86/perfmon_silvermont_events.h
===>  GENERATE HEADER GCCX86/perfmon_skylakeX_events.h
===>  GENERATE HEADER GCCX86/perfmon_skylake_events.h
===>  GENERATE HEADER GCCX86/perfmon_tigerlake_events.h
===>  GENERATE HEADER GCCX86/perfmon_westmereEX_events.h
===>  GENERATE HEADER GCCX86/perfmon_westmere_events.h
===>  GENERATE HEADER GCCX86/perfmon_zen2_events.h
===>  GENERATE HEADER GCCX86/perfmon_zen3_events.h
===>  GENERATE HEADER GCCX86/perfmon_zen_events.h
===>  COMPILE  GCCX86/access.o                                                                                               
===>  COMPILE  GCCX86/access_client.o                                                                                        
===>  COMPILE  GCCX86/access_x86.o                                                                                           
===>  COMPILE  GCCX86/access_x86_clientmem.o                                                                                 
===>  COMPILE  GCCX86/access_x86_mmio.o                                                                                      
===>  COMPILE  GCCX86/access_x86_msr.o                                                                                       
===>  COMPILE  GCCX86/access_x86_pci.o
===>  COMPILE  GCCX86/access_x86_rdpmc.o
===>  COMPILE  GCCX86/affinity.o
===>  COMPILE  GCCX86/bitUtil.o 
===>  COMPILE  GCCX86/bstrlib.o 
===>  COMPILE  GCCX86/bstrlib_helper.o
===>  COMPILE  GCCX86/calculator.o
===>  COMPILE  GCCX86/calculator_stack.o
===>  COMPILE  GCCX86/configuration.o
===>  COMPILE  GCCX86/cpuFeatures.o
===>  COMPILE  GCCX86/cpustring.o
===>  COMPILE  GCCX86/frequency_cpu.o
===>  COMPILE  GCCX86/frequency_uncore.o
===>  COMPILE  GCCX86/ghash.o
In file included from ./src/ghash.c:35:0:
./src/ghash.c: In function ‘g_direct_hash’:
./src/includes/ghash.h:37:39: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
 # define GPOINTER_TO_UINT(p) ((guint) (guint64) (p))
                                       ^
./src/ghash.c:792:10: note: in expansion of macro ‘GPOINTER_TO_UINT’
   return GPOINTER_TO_UINT (v); 
          ^
===>  COMPILE  GCCX86/hashTable.o
===>  COMPILE  GCCX86/libperfctr.o
===>  COMPILE  GCCX86/luawid.o
===>  COMPILE  GCCX86/map.o
===>  COMPILE  GCCX86/memsweep.o
===>  COMPILE  GCCX86/numa.o
===>  COMPILE  GCCX86/numa_hwloc.o
===>  COMPILE  GCCX86/numa_proc.o
===>  COMPILE  GCCX86/numa_virtual.o
===>  COMPILE  GCCX86/pci_hwloc.o
===>  COMPILE  GCCX86/pci_proc.o                                                                                             
===>  COMPILE  GCCX86/perfgroup.o                                                                                            
===>  COMPILE  GCCX86/perfmon.o                                                                                              
===>  COMPILE  GCCX86/power.o                                                                                                
./src/power.c: In function ‘power_init’:                                                                                     
./src/power.c:248:25: warning: passing argument 4 of ‘HPMread’ from incompatible pointer type [enabled by default]           
                         err = HPMread(cpuId, MSR_DEV, MSR_TURBO_RATIO_LIMIT1, &flag_vals[1]);                               
                         ^                                                                                                   
In file included from ./src/includes/power.h:39:0,
                 from ./src/power.c:39:
./src/includes/access.h:38:5: note: expected ‘uint64_t *’ but argument is of type ‘long unsigned int *’
 int HPMread(int cpu_id, PciDeviceIndex dev, uint32_t reg, uint64_t* data);
     ^
./src/power.c:256:25: warning: passing argument 4 of ‘HPMread’ from incompatible pointer type [enabled by default]
                         err = HPMread(cpuId, MSR_DEV, MSR_TURBO_RATIO_LIMIT2, &flag_vals[2]);
                         ^
In file included from ./src/includes/power.h:39:0,
                 from ./src/power.c:39:
./src/includes/access.h:38:5: note: expected ‘uint64_t *’ but argument is of type ‘long unsigned int *’
 int HPMread(int cpu_id, PciDeviceIndex dev, uint32_t reg, uint64_t* data);
     ^
./src/power.c:264:25: warning: passing argument 4 of ‘HPMread’ from incompatible pointer type [enabled by default]
                         err = HPMread(cpuId, MSR_DEV, MSR_TURBO_RATIO_LIMIT3, &flag_vals[3]);
                         ^
In file included from ./src/includes/power.h:39:0,
                 from ./src/power.c:39:
./src/includes/access.h:38:5: note: expected ‘uint64_t *’ but argument is of type ‘long unsigned int *’
 int HPMread(int cpu_id, PciDeviceIndex dev, uint32_t reg, uint64_t* data);
     ^
===>  COMPILE  GCCX86/thermal.o 
===>  COMPILE  GCCX86/timer.o
===>  COMPILE  GCCX86/topology.o
===>  COMPILE  GCCX86/topology_cpuid.o
===>  COMPILE  GCCX86/topology_hwloc.o
===>  COMPILE  GCCX86/topology_proc.o
===>  COMPILE  GCCX86/tree.o
===>  COMPILE  GCCX86/voltage.o 
===>  COMPILE  GCCX86/loadData.o
===>  EXTERNAL HWLOC
===>  EXTERNAL LUA
===>  CREATE SHARED LIB  liblikwid.so
===>  CREATE LIB  liblikwidpin.so
make[1]: Entering directory '/workspace/srcdir/likwid-5.2.1/src/pthread-overload'
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
make[1]: Leaving directory '/workspace/srcdir/likwid-5.2.1/src/pthread-overload'
===>  ADJUSTING  likwid-perfctr
===>  ADJUSTING  likwid-pin
===>  ADJUSTING  likwid-powermeter
===>  ADJUSTING  likwid-topology
===>  ADJUSTING  likwid-memsweeper
===>  ADJUSTING  likwid-mpirun
===>  ADJUSTING  likwid-features
===>  ADJUSTING  likwid-perfscope
===>  ADJUSTING  likwid-genTopoCfg
===>  ADJUSTING  likwid.lua
===>  ENTER  bench
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
===>  COMPILE C GCCX86/allocator.o
===>  COMPILE C GCCX86/barrier.o
===>  COMPILE C GCCX86/bench.o
===>  COMPILE C GCCX86/bstrlib.o
===>  COMPILE C GCCX86/bstrlib_helper.o
===>  COMPILE C GCCX86/ptt2asm.o
===>  COMPILE C GCCX86/strUtil.o
===>  COMPILE C GCCX86/threads.o
===>  GENERATE BENCHMARKS
===>  ASSEMBLE  GCCX86/copy.o
===>  ASSEMBLE  GCCX86/load.o
===>  ASSEMBLE  GCCX86/store.o
===>  ASSEMBLE  GCCX86/stream.o
===>  LINKING  likwid-bench

May 18 '22 16:05 carstenbauer

How do I set COMPILER = GCCPOWER for powerpc and COMPILER = GCC for regular x86_64 (i.e. system dependent patches)?

if [[ "${target}" == powerpc64le-linux-* ]]; then
    COMPILER=GCCPOWER
elif [[ "${target}" == aarch64-linux-* ]]; then
    COMPILER=GCCARMv8
else # assume x86_64
    COMPILER=GCC
fi

May 18 '22 16:05 giordano

FYI: I'll wait until the next likwid release before continuing on this.

Jun 15 '22 06:06 carstenbauer

I'm in the progress of making LIKWID relocatable but it's not finished yet: https://github.com/RRZE-HPC/likwid/pull/480

Sep 27 '22 10:09 TomTheBear

Is this good to go now?

No it's not (for the reasons you mentioned). Just wanted to push it a bit closer to the finish line.

Sep 27 '22 10:09 carstenbauer

This would be an awesome feature to have. Looking forward to the progress that @TomTheBear can make on this.

Dec 27 '22 22:12 miguelraz

This is not an easy task. LIKWID consists of multiple files (misc Lua functions, C library, performance group files, benchmark files, accessdaemon, ...) that need to be found at runtime and interact with each other (e.g. UNIX socket communication).

Dec 30 '22 12:12 TomTheBear

Yggdrasil Yggdrasil copied to clipboard

[WIP] Add likwid 5.2.1

Yggdrasil
Yggdrasil copied to clipboard