Yggdrasil
Yggdrasil copied to clipboard
[WIP] Add likwid 5.2.1
This is my attempt to provide a basic configuration of LIKWID, in particular the library liblikwid
, via BinaryBuilder (https://github.com/JuliaPerf/LIKWID.jl/issues/24). The plan is to later use this as a fallback in LIKWID.jl (if there is no system-wide likwid available).
TODOs:
- [x] Reuse hwloc and Lua JLLs
- [x] use a patch instead of sed
- [x] fix
ldd: --version
issues (as much as possible)
Potential TODOs:
- [ ] provide the executable lua scripts (e.g.
likwid-topology
,likwid-perfctr
, etc.) asFileProducts
-> difficult due to absolute paths - [ ] support for musl (if possible)
- [ ] support for aarch64 and powerpc?
Questions / Remarks:
- To modify the
config.mk
, I've opted for threesed
commands. (Should I use a patch instead? What would be the advantage / disadvantage?) - I get the following
make
log:
sandbox:${WORKSPACE}/srcdir/likwid-5.2.1 # make
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
===> GENERATE HEADER GCC/perfmon_a15_events.h
===> GENERATE HEADER GCC/perfmon_a57_events.h
===> GENERATE HEADER GCC/perfmon_a64fx_events.h
[ ... many more GENERATE HEADERs ... ]
===> GENERATE HEADER GCC/perfmon_zen2_events.h
===> GENERATE HEADER GCC/perfmon_zen3_events.h
===> GENERATE HEADER GCC/perfmon_zen_events.h
===> COMPILE GCC/access.o
===> COMPILE GCC/access_client.o
===> COMPILE GCC/access_x86.o
[ ... many more COMPILEs ...]
===> COMPILE GCC/tree.o
===> COMPILE GCC/voltage.o
===> COMPILE GCC/loadData.o
===> ENTER /workspace/srcdir/likwid-5.2.1/ext/hwloc
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
===> ENTER /workspace/srcdir/likwid-5.2.1/ext/lua
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
===> CREATE SHARED LIB liblikwid.so
===> CREATE LIB liblikwidpin.so
make[1]: Entering directory '/workspace/srcdir/likwid-5.2.1/src/pthread-overload'
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
make[1]: Leaving directory '/workspace/srcdir/likwid-5.2.1/src/pthread-overload'
===> ADJUSTING likwid-perfctr
===> ADJUSTING likwid-pin
===> ADJUSTING likwid-powermeter
===> ADJUSTING likwid-topology
===> ADJUSTING likwid-memsweeper
===> ADJUSTING likwid-mpirun
===> ADJUSTING likwid-features
===> ADJUSTING likwid-perfscope
===> ADJUSTING likwid-genTopoCfg
===> ADJUSTING likwid.lua
===> ENTER bench
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
===> COMPILE C GCC/allocator.o
===> COMPILE C GCC/barrier.o
===> COMPILE C GCC/bench.o
===> COMPILE C GCC/bstrlib.o
===> COMPILE C GCC/bstrlib_helper.o
===> COMPILE C GCC/ptt2asm.o
===> COMPILE C GCC/strUtil.o
===> COMPILE C GCC/threads.o
===> GENERATE BENCHMARKS
===> ASSEMBLE GCC/clcopy.o
===> ASSEMBLE GCC/clload.o
[ ... many more ASSEMBLEs ...]
===> ASSEMBLE GCC/update_sp_sse.o
===> ASSEMBLE GCC/update_sse.o
===> LINKING likwid-bench
Note the
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
bits. Should I care about those given that the build process seems to proceed just fine?
(@TomTheBear, @Suavesito-Olimpiada FYI)
Patch tends to be more stable since it will fail if it couldn't apply it.
Ideally you would be reusing HWLoc_jll instead of building it again, same with lua
@carstenbauer: The ldd --version
call is used to determine the glibc version because old glibc version lack the wrappers for sched_setaffinity
. You can just add -DHAS_SCHEDAFFINITY
to the DEFINES
to forcefully use the wrapper (which should be available on all systems nowadays).
I never tried building LIKWID with musl.
For building with external hwloc and lua, check spack's LIKWID package.
Ideally you would be reusing HWLoc_jll instead of building it again, same with lua
Yes, indeed, just forgot to put it on the Todo list.
You can just add
-DHAS_SCHEDAFFINITY
to theDEFINES
to forcefully use the wrapper (which should be available on all systems nowadays).
@TomTheBear Can you elaborate a bit more, i.e. where should I add it exactly? I tried setting it in make/config_defines.mk
(and make/include_GCC.mk
/ make/include_GCCX86.mk
) without success.
Can you elaborate a bit more, i.e. where should I add it exactly? I tried setting it in make/config_defines.mk (and make/include_GCC.mk / make/include_GCCX86.mk) without success.
If you add it to the DEFINES
in make/include_<COMPILER>.mk
, it is present on the compile lines. You can check with make Q=
:
$ make Q=
===> COMPILE GCC/perfmon.o
gcc [...] -DHAS_SCHEDAFFINITY [...]
@TomTheBear That's what I did (and it works in the sense that you describe) but I still get
sandbox:${WORKSPACE}/srcdir/likwid-5.2.1 # make Q=
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
===> GENERATE HEADER GCC/perfmon_a15_events.h
./perl/gen_events.pl src/includes/perfmon_a15_events.txt GCC/perfmon_a15_events.h
===> GENERATE HEADER GCC/perfmon_a57_events.h
./perl/gen_events.pl src/includes/perfmon_a57_events.txt GCC/perfmon_a57_events.h
===> GENERATE HEADER GCC/perfmon_a64fx_events.h
./perl/gen_events.pl src/includes/perfmon_a64fx_events.txt GCC/perfmon_a64fx_events.h
===> GENERATE HEADER GCC/perfmon_atom_events.h
./perl/gen_events.pl src/includes/perfmon_atom_events.txt GCC/perfmon_atom_events.h
===> GENERATE HEADER GCC/perfmon_broadwellEP_events.h
./perl/gen_events.pl src/includes/perfmon_broadwellEP_events.txt GCC/perfmon_broadwellEP_events.h
===> GENERATE HEADER GCC/perfmon_broadwell_events.h
./perl/gen_events.pl src/includes/perfmon_broadwell_events.txt GCC/perfmon_broadwell_events.h
===> GENERATE HEADER GCC/perfmon_broadwelld_events.h
./perl/gen_events.pl src/includes/perfmon_broadwelld_events.txt GCC/perfmon_broadwelld_events.h
===> GENERATE HEADER GCC/perfmon_cascadelakeX_events.h
./perl/gen_events.pl src/includes/perfmon_cascadelakeX_events.txt GCC/perfmon_cascadelakeX_events.h
===> GENERATE HEADER GCC/perfmon_cavtx2_events.h
./perl/gen_events.pl src/includes/perfmon_cavtx2_events.txt GCC/perfmon_cavtx2_events.h
===> GENERATE HEADER GCC/perfmon_core2_events.h
./perl/gen_events.pl src/includes/perfmon_core2_events.txt GCC/perfmon_core2_events.h
===> GENERATE HEADER GCC/perfmon_goldmont_events.h
./perl/gen_events.pl src/includes/perfmon_goldmont_events.txt GCC/perfmon_goldmont_events.h
===> GENERATE HEADER GCC/perfmon_haswellEP_events.h
./perl/gen_events.pl src/includes/perfmon_haswellEP_events.txt GCC/perfmon_haswellEP_events.h
===> GENERATE HEADER GCC/perfmon_haswell_events.h
./perl/gen_events.pl src/includes/perfmon_haswell_events.txt GCC/perfmon_haswell_events.h
===> GENERATE HEADER GCC/perfmon_icelakeX_events.h
./perl/gen_events.pl src/includes/perfmon_icelakeX_events.txt GCC/perfmon_icelakeX_events.h
===> GENERATE HEADER GCC/perfmon_icelake_events.h
./perl/gen_events.pl src/includes/perfmon_icelake_events.txt GCC/perfmon_icelake_events.h
===> GENERATE HEADER GCC/perfmon_interlagos_events.h
./perl/gen_events.pl src/includes/perfmon_interlagos_events.txt GCC/perfmon_interlagos_events.h
===> GENERATE HEADER GCC/perfmon_ivybridgeEP_events.h
./perl/gen_events.pl src/includes/perfmon_ivybridgeEP_events.txt GCC/perfmon_ivybridgeEP_events.h
===> GENERATE HEADER GCC/perfmon_ivybridge_events.h
===> GENERATE HEADER GCC/perfmon_k10_events.h
./perl/gen_events.pl src/includes/perfmon_k10_events.txt GCC/perfmon_k10_events.h
===> GENERATE HEADER GCC/perfmon_k8_events.h
./perl/gen_events.pl src/includes/perfmon_k8_events.txt GCC/perfmon_k8_events.h
===> GENERATE HEADER GCC/perfmon_kabini_events.h
./perl/gen_events.pl src/includes/perfmon_kabini_events.txt GCC/perfmon_kabini_events.h
===> GENERATE HEADER GCC/perfmon_knl_events.h
./perl/gen_events.pl src/includes/perfmon_knl_events.txt GCC/perfmon_knl_events.h
===> GENERATE HEADER GCC/perfmon_nehalemEX_events.h
./perl/gen_events.pl src/includes/perfmon_nehalemEX_events.txt GCC/perfmon_nehalemEX_events.h
===> GENERATE HEADER GCC/perfmon_nehalem_events.h
./perl/gen_events.pl src/includes/perfmon_nehalem_events.txt GCC/perfmon_nehalem_events.h
===> GENERATE HEADER GCC/perfmon_neon1_events.h
./perl/gen_events.pl src/includes/perfmon_neon1_events.txt GCC/perfmon_neon1_events.h
===> GENERATE HEADER GCC/perfmon_p6_events.h
./perl/gen_events.pl src/includes/perfmon_p6_events.txt GCC/perfmon_p6_events.h
===> GENERATE HEADER GCC/perfmon_phi_events.h
./perl/gen_events.pl src/includes/perfmon_phi_events.txt GCC/perfmon_phi_events.h
===> GENERATE HEADER GCC/perfmon_pm_events.h
./perl/gen_events.pl src/includes/perfmon_pm_events.txt GCC/perfmon_pm_events.h
===> GENERATE HEADER GCC/perfmon_power8_events.h
./perl/gen_events.pl src/includes/perfmon_power8_events.txt GCC/perfmon_power8_events.h
===> GENERATE HEADER GCC/perfmon_power9_events.h
./perl/gen_events.pl src/includes/perfmon_power9_events.txt GCC/perfmon_power9_events.h
===> GENERATE HEADER GCC/perfmon_sandybridgeEP_events.h
./perl/gen_events.pl src/includes/perfmon_sandybridgeEP_events.txt GCC/perfmon_sandybridgeEP_events.h
===> GENERATE HEADER GCC/perfmon_sandybridge_events.h
./perl/gen_events.pl src/includes/perfmon_sandybridge_events.txt GCC/perfmon_sandybridge_events.h
===> GENERATE HEADER GCC/perfmon_silvermont_events.h
./perl/gen_events.pl src/includes/perfmon_silvermont_events.txt GCC/perfmon_silvermont_events.h
===> GENERATE HEADER GCC/perfmon_skylakeX_events.h
./perl/gen_events.pl src/includes/perfmon_skylakeX_events.txt GCC/perfmon_skylakeX_events.h
===> GENERATE HEADER GCC/perfmon_skylake_events.h
./perl/gen_events.pl src/includes/perfmon_skylake_events.txt GCC/perfmon_skylake_events.h
===> GENERATE HEADER GCC/perfmon_tigerlake_events.h
./perl/gen_events.pl src/includes/perfmon_tigerlake_events.txt GCC/perfmon_tigerlake_events.h
===> GENERATE HEADER GCC/perfmon_westmereEX_events.h
./perl/gen_events.pl src/includes/perfmon_westmereEX_events.txt GCC/perfmon_westmereEX_events.h
===> GENERATE HEADER GCC/perfmon_westmere_events.h
./perl/gen_events.pl src/includes/perfmon_westmere_events.txt GCC/perfmon_westmere_events.h
===> GENERATE HEADER GCC/perfmon_zen2_events.h
./perl/gen_events.pl src/includes/perfmon_zen2_events.txt GCC/perfmon_zen2_events.h
===> GENERATE HEADER GCC/perfmon_zen3_events.h
./perl/gen_events.pl src/includes/perfmon_zen3_events.txt GCC/perfmon_zen3_events.h
===> GENERATE HEADER GCC/perfmon_zen_events.h
./perl/gen_events.pl src/includes/perfmon_zen_events.txt GCC/perfmon_zen_events.h
===> COMPILE GCC/access.o
gcc -c -O2 -std=c99 -Wno-format -fPIC -fPIC -fvisibility=hidden -DPAGE_ALIGNMENT=4096 -DLIKWID_MONITOR_LOCK -DDEBUGLEV=0 -DHAS_SCHEDAFFINITY -DVERSION=5 -DRELEASE=2 -DMINORVERSION=1 -DCFGFILE=/etc/likwid.cfg -DTOPOFILE=/etc/likwid_topo.cfg -DINSTALL_PREFIX=/workspace/destdir -DMAX_NUM_THREADS=300 -DMAX_NUM_NODES=64 -DMAX_NUM_CLIARGS=16384 -DACCESSDAEMON=/workspace/destdir/sbin/likwid-accessD -DFREQDAEMON=/workspace/destdir/sbin/likwid-setFreq -DGROUPPATH=/workspace/destdir/share/likwid/perfgroups -DLIKWIDLOCK=/var/run/likwid.lock -DLIKWIDSOCKETBASE=/tmp/likwid -DGITCOMMIT=233ab943543480cd46058b34616c174198ba0459 -D_GNU_SOURCE -DCOLOR=BLUE -DLIKWID_USE_PERFEVENT -DHAS_MEMPOLICY -DHAS_SCHEDAFFINITY -DLIKWID_USE_HWLOC -DLIKWID_USE_PERFEVENT -DACCESSMODE=-1 -I./src/includes -I/workspace/srcdir/likwid-5.2.1/ext/lua/includes -I/workspace/srcdir/likwid-5.2.1/ext/hwloc/include -I./GCC ./src/access.c -o GCC/access.o
[....]
Do I need to put it somewhere else as well?
The ldd --version call is used to determine the glibc version because old glibc version lack the wrappers for sched_setaffinity.
sandbox:${WORKSPACE} # echo '#include <features.h>' | cc -dM -E - | grep -E '#define __GLIBC_(|MINOR_)_'
#define __GLIBC__ 2
#define __GLIBC_MINOR__ 12
is probably more cross-compilation friendly than running ldd
on a system where ldd
is not from glibc.
Related to the topic of cross-compilation, we looked into building likwid some time ago, but I remember it wasn't really possible to do a cross-build because the package needed some information about the target system at compile time, or some local kernel modules, or something like that. Is this still the case?
For building with external hwloc and lua, check spack's LIKWID package.
It seems like likwid still installs libhwloc.so.5.2
(and makes the libhwloc.so
symlink point to it) when I specify an external hwloc. This is the $libdir
before and after make install [...]
:
Before
lrwxrwxrwx 1 root root 18 May 15 17:31 libhwloc.so -> libhwloc.so.15.5.3*
lrwxrwxrwx 1 root root 18 May 15 17:31 libhwloc.so.15 -> libhwloc.so.15.5.3*
lrwxrwxrwx 1 root root 79 May 15 17:31 libhwloc.so.15.5.3 -> ../../artifacts/f11a525a0a63e8dac0e0236c0e578c0d8e34d59e/l
ib/libhwloc.so.15.5.3*
After
lrwxrwxrwx 1 root root 15 May 15 17:52 libhwloc.so -> libhwloc.so.5.2*
lrwxrwxrwx 1 root root 18 May 15 17:31 libhwloc.so.15 -> libhwloc.so.15.5.3*
lrwxrwxrwx 1 root root 79 May 15 17:31 libhwloc.so.15.5.3 -> ../../artifacts/f11a525a0a63e8dac0e0236c0e578c0d8e34d59e/l
ib/libhwloc.so.15.5.3*
lrwxrwxrwx 1 root root 15 May 15 17:52 libhwloc.so.5 -> libhwloc.so.5.2*
-rwxr-xr-x 1 root root 1729301 May 15 17:52 libhwloc.so.5.2*
I see that they apply a patch in the easyconfig that addresses this. @TomTheBear Why isn't the easyconfig patch the default? I.e., is there a deeper reason for the creation of libhwloc.so.5.2
? Otherwise I'd apply a similar patch.
The script bin/likwid-topology
(and probably also the others) hardcodes local absolute paths of the build environment, which won't exist at runtime. That's not going to work.
The script
bin/likwid-topology
(and probably also the others) hardcodes local absolute paths of the build environment, which won't exist at runtime. That's not going to work.
Yeah. That's certainly unfortunate. But the liblikwid.so
is the primary reason for me to attempt to create a JLL. You can use lots (most?) of the features without the command-line tools. (One could argue that the "in-Julia" interface to LIKWID is anyways nicer / more julian than the CLI.)
support for musl (if possible)
It'd be nice to build for all Linux platforms, if possible (at least glibc ones, if that's a requirement). I'm thinking in particular of aarch64 and powerpc64le, which are more popular in HPC settings.
Why isn't the easyconfig patch the default? I.e., is there a deeper reason for the creation of libhwloc.so.5.2?
@carstenbauer: I test LIKWID only in the default configuration varying important options (COMPILER
and ACCESSMODE
). The feature to use external lua and hwloc was only added for the package management folks out there. The reason is quite simple, there is not much feedback from the package management folks. I didn't even know that an easy_build recipe exists for LIKWID. And consequently, I didn't know the patch. The library is probably not needed as the easy_build guys (hopefully) tested their stuff.
The script bin/likwid-topology(and probably also the others) hardcodes local absolute paths of the build environment, which won't exist at runtime. That's not going to work.
@giordano This can be avoided by getting the script path in the beginning. It's just Lua. Patch it if you need it differently. Until now, Yggdrasil seems to be the only package manager having problems with absolute paths.
Until now, Yggdrasil seems to be the only package manager having problems with absolute paths.
This is the only sensible thing given that Yggdrasil
- does not have administrator permissions on systems,
- is cross-compiled on a Linux system,
- want to support multiple distribution and versions of different OS's,
- and must be relocatable.
The last point might be the most important of all.
This is the only sensible thing given that Yggdrasil [...]
While true, that's a Julia / Yggdrasil perspective. A tool (like likwid) may not care about what Yggdrasil wants / needs. It's a tedious task to cater to the specific needs of various package managers so we can't expect non-Julia developers to spend much time on it. We can hope though 😉
Fortunately, in this case, @TomTheBear has already committed a few simple patches to likwid so that from version 5.2.2 on
- the
ldd --version
issue should be gone (https://github.com/RRZE-HPC/likwid/commit/d6bbdcf2a84210d9c1dd6952d1ae533ae36bff07) and - we don't need to patch the Makefile any more (i.e. we can drop the easyconfig-like patch) (https://github.com/RRZE-HPC/likwid/commit/3a79b059c57f03949a74977b654a416b5ed51fcf)
Thanks for this! ❤️
A tool (like likwid) may not care about what Yggdrasil wants / needs.
Absolutely, I just wanted to explain the perspective of Julia/Yggdrasil, not to rant about the projects that don't do it, or ask for every project to do it. I'm sorry if my comment read otherwise. :sweat_smile:
Fortunately, in this case, @TomTheBear has already committed a few simple patches to likwid
Thanks, @TomTheBear! That sounds incredible. :tada:
It'd be nice to build for all Linux platforms, if possible (at least glibc ones, if that's a requirement). I'm thinking in particular of aarch64 and powerpc64le, which are more popular in HPC settings.
PowerPC
FWIW, if I set COMPILER = GCCPOWER
in config.mk
(and use external hwloc and lua) compilation works without errors. Q: How do I set COMPILER = GCCPOWER
for powerpc and COMPILER = GCC
for regular x86_64 (i.e. system dependent patches)?
aarch64
COMPILER = GCCARMv8
doesn't work:
sandbox:${WORKSPACE}/srcdir/likwid-5.2.1 # make PREFIX=${prefix} HWLOC_INCLUDE_DIR=${includedir} HWLOC_LIB_DIR=${libdir} HWLOC_LIB_NAME=hwloc LUA_INCLUDE_DIR=${includedir} LUA_LIB_DIR=${libdir} LUA_LIB_NAME=lua LUA_BIN=${bindir}
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
===> GENERATE HEADER GCCARMv8/perfmon_a15_events.h
===> GENERATE HEADER GCCARMv8/perfmon_a57_events.h
===> GENERATE HEADER GCCARMv8/perfmon_a64fx_events.h
===> GENERATE HEADER GCCARMv8/perfmon_atom_events.h
===> GENERATE HEADER GCCARMv8/perfmon_broadwellEP_events.h
===> GENERATE HEADER GCCARMv8/perfmon_broadwell_events.h
===> GENERATE HEADER GCCARMv8/perfmon_broadwelld_events.h
===> GENERATE HEADER GCCARMv8/perfmon_cascadelakeX_events.h
===> GENERATE HEADER GCCARMv8/perfmon_cavtx2_events.h
===> GENERATE HEADER GCCARMv8/perfmon_core2_events.h
===> GENERATE HEADER GCCARMv8/perfmon_goldmont_events.h
===> GENERATE HEADER GCCARMv8/perfmon_haswellEP_events.h
===> GENERATE HEADER GCCARMv8/perfmon_haswell_events.h
===> GENERATE HEADER GCCARMv8/perfmon_icelakeX_events.h
===> GENERATE HEADER GCCARMv8/perfmon_icelake_events.h
===> GENERATE HEADER GCCARMv8/perfmon_interlagos_events.h
===> GENERATE HEADER GCCARMv8/perfmon_ivybridgeEP_events.h
===> GENERATE HEADER GCCARMv8/perfmon_ivybridge_events.h
===> GENERATE HEADER GCCARMv8/perfmon_k10_events.h
===> GENERATE HEADER GCCARMv8/perfmon_k8_events.h
===> GENERATE HEADER GCCARMv8/perfmon_kabini_events.h
===> GENERATE HEADER GCCARMv8/perfmon_knl_events.h
===> GENERATE HEADER GCCARMv8/perfmon_nehalemEX_events.h
===> GENERATE HEADER GCCARMv8/perfmon_nehalem_events.h
===> GENERATE HEADER GCCARMv8/perfmon_neon1_events.h
===> GENERATE HEADER GCCARMv8/perfmon_p6_events.h
===> GENERATE HEADER GCCARMv8/perfmon_phi_events.h
===> GENERATE HEADER GCCARMv8/perfmon_pm_events.h
===> GENERATE HEADER GCCARMv8/perfmon_power8_events.h
===> GENERATE HEADER GCCARMv8/perfmon_power9_events.h
===> GENERATE HEADER GCCARMv8/perfmon_sandybridgeEP_events.h
===> GENERATE HEADER GCCARMv8/perfmon_sandybridge_events.h
===> GENERATE HEADER GCCARMv8/perfmon_silvermont_events.h
===> GENERATE HEADER GCCARMv8/perfmon_skylakeX_events.h
===> GENERATE HEADER GCCARMv8/perfmon_skylake_events.h
===> GENERATE HEADER GCCARMv8/perfmon_tigerlake_events.h
===> GENERATE HEADER GCCARMv8/perfmon_westmereEX_events.h
===> GENERATE HEADER GCCARMv8/perfmon_westmere_events.h
===> GENERATE HEADER GCCARMv8/perfmon_zen2_events.h
===> GENERATE HEADER GCCARMv8/perfmon_zen3_events.h
===> GENERATE HEADER GCCARMv8/perfmon_zen_events.h
===> COMPILE GCCARMv8/access.o
BinaryBuilder: Cannot force an architecture via -march
make: *** [Makefile:302: GCCARMv8/access.o] Error 1
i686
COMPILER = GCCX86
works (with warnings)
sandbox:${WORKSPACE}/srcdir/likwid-5.2.1 # make PREFIX=${prefix} HWLOC_INCLUDE_DIR=${includedir} HWLOC_LIB_DIR=${libdir} HWLO
C_LIB_NAME=hwloc LUA_INCLUDE_DIR=${includedir} LUA_LIB_DIR=${libdir} LUA_LIB_NAME=lua LUA_BIN=${bindir}
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
===> GENERATE HEADER GCCX86/perfmon_a15_events.h
===> GENERATE HEADER GCCX86/perfmon_a57_events.h
===> GENERATE HEADER GCCX86/perfmon_a64fx_events.h
===> GENERATE HEADER GCCX86/perfmon_atom_events.h
===> GENERATE HEADER GCCX86/perfmon_broadwellEP_events.h
===> GENERATE HEADER GCCX86/perfmon_broadwell_events.h
===> GENERATE HEADER GCCX86/perfmon_broadwelld_events.h
===> GENERATE HEADER GCCX86/perfmon_cascadelakeX_events.h
===> GENERATE HEADER GCCX86/perfmon_cavtx2_events.h
===> GENERATE HEADER GCCX86/perfmon_core2_events.h
===> GENERATE HEADER GCCX86/perfmon_goldmont_events.h
===> GENERATE HEADER GCCX86/perfmon_haswellEP_events.h
===> GENERATE HEADER GCCX86/perfmon_haswell_events.h
===> GENERATE HEADER GCCX86/perfmon_icelakeX_events.h
===> GENERATE HEADER GCCX86/perfmon_icelake_events.h
===> GENERATE HEADER GCCX86/perfmon_interlagos_events.h
===> GENERATE HEADER GCCX86/perfmon_ivybridgeEP_events.h
===> GENERATE HEADER GCCX86/perfmon_ivybridge_events.h
===> GENERATE HEADER GCCX86/perfmon_k10_events.h
===> GENERATE HEADER GCCX86/perfmon_k8_events.h
===> GENERATE HEADER GCCX86/perfmon_kabini_events.h
===> GENERATE HEADER GCCX86/perfmon_knl_events.h
===> GENERATE HEADER GCCX86/perfmon_nehalemEX_events.h
===> GENERATE HEADER GCCX86/perfmon_nehalem_events.h
===> GENERATE HEADER GCCX86/perfmon_neon1_events.h
===> GENERATE HEADER GCCX86/perfmon_p6_events.h
===> GENERATE HEADER GCCX86/perfmon_phi_events.h
===> GENERATE HEADER GCCX86/perfmon_pm_events.h
===> GENERATE HEADER GCCX86/perfmon_power8_events.h
===> GENERATE HEADER GCCX86/perfmon_power9_events.h
===> GENERATE HEADER GCCX86/perfmon_sandybridgeEP_events.h
===> GENERATE HEADER GCCX86/perfmon_sandybridge_events.h
===> GENERATE HEADER GCCX86/perfmon_silvermont_events.h
===> GENERATE HEADER GCCX86/perfmon_skylakeX_events.h
===> GENERATE HEADER GCCX86/perfmon_skylake_events.h
===> GENERATE HEADER GCCX86/perfmon_tigerlake_events.h
===> GENERATE HEADER GCCX86/perfmon_westmereEX_events.h
===> GENERATE HEADER GCCX86/perfmon_westmere_events.h
===> GENERATE HEADER GCCX86/perfmon_zen2_events.h
===> GENERATE HEADER GCCX86/perfmon_zen3_events.h
===> GENERATE HEADER GCCX86/perfmon_zen_events.h
===> COMPILE GCCX86/access.o
===> COMPILE GCCX86/access_client.o
===> COMPILE GCCX86/access_x86.o
===> COMPILE GCCX86/access_x86_clientmem.o
===> COMPILE GCCX86/access_x86_mmio.o
===> COMPILE GCCX86/access_x86_msr.o
===> COMPILE GCCX86/access_x86_pci.o
===> COMPILE GCCX86/access_x86_rdpmc.o
===> COMPILE GCCX86/affinity.o
===> COMPILE GCCX86/bitUtil.o
===> COMPILE GCCX86/bstrlib.o
===> COMPILE GCCX86/bstrlib_helper.o
===> COMPILE GCCX86/calculator.o
===> COMPILE GCCX86/calculator_stack.o
===> COMPILE GCCX86/configuration.o
===> COMPILE GCCX86/cpuFeatures.o
===> COMPILE GCCX86/cpustring.o
===> COMPILE GCCX86/frequency_cpu.o
===> COMPILE GCCX86/frequency_uncore.o
===> COMPILE GCCX86/ghash.o
In file included from ./src/ghash.c:35:0:
./src/ghash.c: In function ‘g_direct_hash’:
./src/includes/ghash.h:37:39: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
# define GPOINTER_TO_UINT(p) ((guint) (guint64) (p))
^
./src/ghash.c:792:10: note: in expansion of macro ‘GPOINTER_TO_UINT’
return GPOINTER_TO_UINT (v);
^
===> COMPILE GCCX86/hashTable.o
===> COMPILE GCCX86/libperfctr.o
===> COMPILE GCCX86/luawid.o
===> COMPILE GCCX86/map.o
===> COMPILE GCCX86/memsweep.o
===> COMPILE GCCX86/numa.o
===> COMPILE GCCX86/numa_hwloc.o
===> COMPILE GCCX86/numa_proc.o
===> COMPILE GCCX86/numa_virtual.o
===> COMPILE GCCX86/pci_hwloc.o
===> COMPILE GCCX86/pci_proc.o
===> COMPILE GCCX86/perfgroup.o
===> COMPILE GCCX86/perfmon.o
===> COMPILE GCCX86/power.o
./src/power.c: In function ‘power_init’:
./src/power.c:248:25: warning: passing argument 4 of ‘HPMread’ from incompatible pointer type [enabled by default]
err = HPMread(cpuId, MSR_DEV, MSR_TURBO_RATIO_LIMIT1, &flag_vals[1]);
^
In file included from ./src/includes/power.h:39:0,
from ./src/power.c:39:
./src/includes/access.h:38:5: note: expected ‘uint64_t *’ but argument is of type ‘long unsigned int *’
int HPMread(int cpu_id, PciDeviceIndex dev, uint32_t reg, uint64_t* data);
^
./src/power.c:256:25: warning: passing argument 4 of ‘HPMread’ from incompatible pointer type [enabled by default]
err = HPMread(cpuId, MSR_DEV, MSR_TURBO_RATIO_LIMIT2, &flag_vals[2]);
^
In file included from ./src/includes/power.h:39:0,
from ./src/power.c:39:
./src/includes/access.h:38:5: note: expected ‘uint64_t *’ but argument is of type ‘long unsigned int *’
int HPMread(int cpu_id, PciDeviceIndex dev, uint32_t reg, uint64_t* data);
^
./src/power.c:264:25: warning: passing argument 4 of ‘HPMread’ from incompatible pointer type [enabled by default]
err = HPMread(cpuId, MSR_DEV, MSR_TURBO_RATIO_LIMIT3, &flag_vals[3]);
^
In file included from ./src/includes/power.h:39:0,
from ./src/power.c:39:
./src/includes/access.h:38:5: note: expected ‘uint64_t *’ but argument is of type ‘long unsigned int *’
int HPMread(int cpu_id, PciDeviceIndex dev, uint32_t reg, uint64_t* data);
^
===> COMPILE GCCX86/thermal.o
===> COMPILE GCCX86/timer.o
===> COMPILE GCCX86/topology.o
===> COMPILE GCCX86/topology_cpuid.o
===> COMPILE GCCX86/topology_hwloc.o
===> COMPILE GCCX86/topology_proc.o
===> COMPILE GCCX86/tree.o
===> COMPILE GCCX86/voltage.o
===> COMPILE GCCX86/loadData.o
===> EXTERNAL HWLOC
===> EXTERNAL LUA
===> CREATE SHARED LIB liblikwid.so
===> CREATE LIB liblikwidpin.so
make[1]: Entering directory '/workspace/srcdir/likwid-5.2.1/src/pthread-overload'
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
make[1]: Leaving directory '/workspace/srcdir/likwid-5.2.1/src/pthread-overload'
===> ADJUSTING likwid-perfctr
===> ADJUSTING likwid-pin
===> ADJUSTING likwid-powermeter
===> ADJUSTING likwid-topology
===> ADJUSTING likwid-memsweeper
===> ADJUSTING likwid-mpirun
===> ADJUSTING likwid-features
===> ADJUSTING likwid-perfscope
===> ADJUSTING likwid-genTopoCfg
===> ADJUSTING likwid.lua
===> ENTER bench
ldd: --version: No such file or directory
sh: 4: unknown operand
Info: Compiling for perf_event interface. Measurements of thermal information is disabled
===> COMPILE C GCCX86/allocator.o
===> COMPILE C GCCX86/barrier.o
===> COMPILE C GCCX86/bench.o
===> COMPILE C GCCX86/bstrlib.o
===> COMPILE C GCCX86/bstrlib_helper.o
===> COMPILE C GCCX86/ptt2asm.o
===> COMPILE C GCCX86/strUtil.o
===> COMPILE C GCCX86/threads.o
===> GENERATE BENCHMARKS
===> ASSEMBLE GCCX86/copy.o
===> ASSEMBLE GCCX86/load.o
===> ASSEMBLE GCCX86/store.o
===> ASSEMBLE GCCX86/stream.o
===> LINKING likwid-bench
How do I set COMPILER = GCCPOWER for powerpc and COMPILER = GCC for regular x86_64 (i.e. system dependent patches)?
if [[ "${target}" == powerpc64le-linux-* ]]; then
COMPILER=GCCPOWER
elif [[ "${target}" == aarch64-linux-* ]]; then
COMPILER=GCCARMv8
else # assume x86_64
COMPILER=GCC
fi
FYI: I'll wait until the next likwid release before continuing on this.
I'm in the progress of making LIKWID relocatable but it's not finished yet: https://github.com/RRZE-HPC/likwid/pull/480
Is this good to go now?
No it's not (for the reasons you mentioned). Just wanted to push it a bit closer to the finish line.
This would be an awesome feature to have. Looking forward to the progress that @TomTheBear can make on this.
This is not an easy task. LIKWID consists of multiple files (misc Lua functions, C library, performance group files, benchmark files, accessdaemon, ...) that need to be found at runtime and interact with each other (e.g. UNIX socket communication).