arm linux 4.19 'could not link map to queue'

[EvanTheB]

I am having some trouble getting ply going on an arm chromebook.

Linux localhost 4.19.184 #29 SMP PREEMPT Mon Apr 12 17:19:19 AEST 2021 aarch64 GNU/Linux

I compiled using the gentoo ebuild: https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-util/ply

Which uses ply-2.1.1 + an arm patch: https://github.com/iovisor/ply/commit/1bc183af8703023e377f92716ecd3f339ffffd11.patch

# ply 'kprobe:do_sys_open { printf("%v(%v): %s\n", comm, uid, str(arg1)); }'
error: could not link map to queue
error: unable to create buffer 'stdbuf'
ERR:-22

Which looks like this issue: https://github.com/iovisor/ply/issues/48 I am not sure the best testcase to run, nothing works that I can see.

Here is the strace snippet:

openat(AT_FDCWD, "/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 3
read(3, "0-3\n", 8192)                  = 4
close(3)                                = 0
access("k:schedule { exit(0); }", R_OK) = -1 ENOENT (No such file or directory)
getpid()                                = 19458
stat64("/var/tmp/ply-ksyms", {st_mode=S_IFREG|0644, st_size=192, ...}) = 0
openat(AT_FDCWD, "/var/tmp/ply-ksyms", O_RDWR) = 3
mmap2(NULL, 192, PROT_READ|PROT_WRITE, MAP_SHARED, 3, 0) = 0xead4c000
stat64("/proc", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
stat64("/var/tmp/ply-ksyms", {st_mode=S_IFREG|0644, st_size=192, ...}) = 0
ugetrlimit(RLIMIT_MEMLOCK, {rlim_cur=64*1024, rlim_max=64*1024}) = 0
prlimit64(0, RLIMIT_MEMLOCK, {rlim_cur=RLIM64_INFINITY, rlim_max=RLIM64_INFINITY}, NULL) = 0
bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_PERF_EVENT_ARRAY, key_size=4, value_size=4, max_entries=4, map_flags=0, inner_map_fd=0, map_name="", map_ifindex=0}, 48) = 4
perf_event_open({type=PERF_TYPE_SOFTWARE, size=0 /* PERF_ATTR_SIZE_??? */, config=PERF_COUNT_SW_BPF_OUTPUT, ...}, -1, 0, -1, 0) = 5
bpf(BPF_MAP_UPDATE_ELEM, {map_fd=4, key=0xffe63d34, value=0x106f57e4, flags=BPF_ANY, ...}, 48) = -1 EINVAL (Invalid argument)
write(2, "error: could not link map to que"..., 35error: could not link map to queue

Let me know what else I can try

config.txt

[EvanTheB]

I also tried HEAD e25c9134b856cc7ffe9f562ff95caf9487d16b59 but I got a build error:

make[3]: Entering directory '/build/hana/tmp/portage/dev-util/ply-2.1.1-r1/work/ply-2.1.1/src/ply' cd /build/hana/tmp/portage/dev-util/ply-2.1.1-r1/work/ply-2.1.1/src/ply &&
armv7a-cros-linux-gnueabihf-ld.lld -r -b binary -o /build/hana/tmp/portage/dev-util/ply-2.1.1-r1/work/ply-2.1.1/src/ply/self-test.o self-test.sh ld.lld: error: target emulation unknown: -m or at least one .o file required make[3]: *** [Makefile:654: self-test.o] Error 1

[EvanTheB]

This is also not working on an AMD chromebook with kernel 5.10, so the problem is probably either in the ebuild or the chromeos build environment. Any clues would be appreciated, I will try to keep digging.

[EvanTheB]

I got kprobe working with a simple test case, and tried the same in ply. Now I get a segfault in libply.so:

# gdb --args ply -d 'p:__arm64_compat_sys_execve { }' 
...
Starting program: /usr/sbin/ply -d p:__arm64_compat_sys_execve\ \{\ \}
error: 
Program received signal SIGSEGV, Segmentation fault.
0xf7f8f00a in node_vfprintxf () from /usr/lib/libply.so.0

openat(AT_FDCWD, "/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 3
read(3, "0-3\n", 8192)                  = 4
close(3)                                = 0
access("p:__arm64_compat_sys_execve { }", R_OK) = -1 ENOENT (No such file or directory)
getpid()                                = 8604
stat64("/var/tmp/ply-ksyms", {st_mode=S_IFREG|0644, st_size=192, ...}) = 0
openat(AT_FDCWD, "/var/tmp/ply-ksyms", O_RDWR) = 3
mmap2(NULL, 192, PROT_READ|PROT_WRITE, MAP_SHARED, 3, 0) = 0xf1a87000
stat64("/proc", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
stat64("/var/tmp/ply-ksyms", {st_mode=S_IFREG|0644, st_size=192, ...}) = 0
write(2, "error: ", 7error: )                  = 7
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x28} ---
+++ killed by SIGSEGV (core dumped) +++
Segmentation fault (core dumped)

[wkz]

Not really sure what to make of the first issue. I do not have easy access to a 4.19/aarch64 system at the moment. What is the output of ply -T if you run from the latest master?

Re: the second issue: p: does not map to any provider in ply, is it copypasta from another tool? That is no excuse for segfaulting of course, just something I noticed. Anyway, I am not able to the reproduce the issue on my aarch64 systems. Maybe you try to figure out how your environment differs from the one used ply's CI (make aarch64-check-install-tc from the test/ directory)

[zhonxinya]

Hi ,I meet the same problem,it may be caused by "using system default linux/bpf.h" whitch not linux 4.19. You may need to change CFLAGS for libply and do below

1. make -C $(KERNEL_DIR) headers_install INSTALL_HDR_PATH=$(TARGET_KERNEL_INSTALL_HEADER) ARCH=arm
2. ./configure --host=arm-linux CC=$(CROSS_COMPILE) CFLAGS="-I$(TARGET_KERNEL_INSTALL_HEADER)/include"
3. make

"KERNEL_DIR" is your kernel tree location and "TARGET_KERNEL_INSTALL_HEADER" is where you install kernel headers.