build error compiling C minimal example - 'make minimal' in examples/c

[dmitris]

When trying to run make minimal in the examples/c directory, getting the following errors - the full output in gist https://gist.github.com/dmitris/697849eb3c0b80e6f2ca50430cae499b:

skeleton/pid_iter.bpf.c:47:14: error: incomplete definition of type 'struct bpf_perf_link'
[...]
skeleton/pid_iter.bpf.c:44:9: note: forward declaration of 'struct bpf_perf_link'
        struct bpf_perf_link *perf_link;
               ^
skeleton/pid_iter.bpf.c:48:10: error: incomplete definition of type 'struct bpf_perf_link'
        event = BPF_CORE_READ(perf_link, perf_file, private_data);
[...]
skeleton/pid_iter.bpf.c:49:30: error: no member named 'bpf_cookie' in 'struct perf_event'
        return BPF_CORE_READ(event, bpf_cookie);

Is it related to (or even fixed in) https://lore.kernel.org/bpf/[email protected]/ and https://lore.kernel.org/bpf/[email protected]/ ?

I got that error with the current rhel8 and Fedora 35 - Linux mybox.example.com 5.14.10-300.fc35.x86_64 #1 SMP Thu Oct 7 20:48:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

[dmitris]

interestingly, on fedora 35 the cmake build (almost) work - I have to compile the bpftool executable in bpftool/src and copy it into the tools subdirectory, otherwise getting this error:

[ 23%] Built target libbpf
[ 41%] Built target libbpf-build
[ 44%] [clang] Building BPF object: bootstrap
[ 47%] [skel]  Building BPF skeleton: bootstrap
bash: line 1: /home/vagrant/gh/libbpf/libbpf-bootstrap/examples/c/../../tools/bpftool: No such file or directory

On rhel8, it doesn't work - skeleton/pid_iter.bpf.c:90:36: error: use of undeclared identifier 'BPF_LINK_TYPE_PERF_EVENT'; did you mean 'BPF_PROG_TYPE_PERF_EVENT'? - the kernel must be too old.

[dmitris]

Never mind - was my bad clone as mentioned in https://github.com/xdp-project/xdp-tutorial/issues/93#issuecomment-1113170389. With the correct clone (git clone --recurse-submodules https://github.com/libbpf/libbpf-bootstrap), the examples/c build is clean on both rhel8 and fc35.

[dmitris]

actually I think this issue needs to be reopened. I carefully followed the instructions for the CMake build from https://github.com/libbpf/libbpf-bootstrap#c-examples (starting with git submodule update --init --recursive), but it does not work - I'm getting the following error:

[ 17%] Built target libbpf-build
[ 20%] [clang] Building BPF object: uprobe
[ 23%] [skel]  Building BPF skeleton: uprobe
bash: /home/dmitris/dev/hack/gh/libbpf/libbpf-bootstrap/examples/c/../../tools/bpftool: No such file or directory
make[2]: *** [CMakeFiles/uprobe.dir/build.make:74: uprobe.skel.h] Error 127
make[2]: *** Deleting file 'uprobe.skel.h'
make[1]: *** [CMakeFiles/Makefile2:93: CMakeFiles/uprobe.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

Do we need to add CMakefile instructions to build bpftool with the correct libbpf code, and copy the binary in the expected place?

[anakryiko]

Please see https://github.com/libbpf/libbpf-bootstrap/issues/70, in case it's related. But also make sure you perform git submodule update --init --recursive (recursive part is important).

[anakryiko]

@dmitris do you still have a problem building examples?

[pynem]

@ankryiko I'm getting this problem when building using the makefile instructions; I also get the error described in #70 if I try to use the cmake instructions.

FYI: I've cloned at commit 2b5b2bb0, and git submodule status --recursive gives: 1e1f48c18da9416e1d4c35ec9bce4ed77019b109 ../../blazesym (1e1f48c) 8ec897a0cd357fe9e13eec7d27d43e024891746b ../../bpftool (v6.8.0-1-g8ec897a) 86eb09863c1c0177e99c2c703092042d3cdba910 ../../bpftool/libbpf (v0.8.0) 4eb6485c08867edaa5a0a81c64ddb23580420340 ../../libbpf (v0.8.0-5-g4eb6485)

OS: Ubuntu 21.04 Kernel: Linux 5.11.0-49-generic x86_64

[algrebe]

@pynem I faced the same problem on Ubuntu 20.04 with 5.13.0-51-generic.

Quoting part of this stackoverflow answer

To compile without error, you need a kernel recent enough to have struct bpf_perf_link defined (it's 5.15+), and compiled with option CONFIG_PERF_EVENTS, as mentioned in the message you found.

For now, undoing the changes in https://github.com/libbpf/bpftool/commit/d97300d3565e99d9fea06d1875935234796f4096 (commenting out/deleting the additions to pid_iter.bpf.c) works as a quick fix. Another way would be to apply the patch that @dmitris referred to.

diff --git a/src/skeleton/pid_iter.bpf.c b/src/skeleton/pid_iter.bpf.c
index eb05ea5..3f81545 100644
--- a/src/skeleton/pid_iter.bpf.c
+++ b/src/skeleton/pid_iter.bpf.c
@@ -38,17 +38,6 @@ static __always_inline __u32 get_obj_id(void *ent, enum bpf_obj_type type)
        }
 }
 
-/* could be used only with BPF_LINK_TYPE_PERF_EVENT links */
-static __u64 get_bpf_cookie(struct bpf_link *link)
-{
-       struct bpf_perf_link *perf_link;
-       struct perf_event *event;
-
-       perf_link = container_of(link, struct bpf_perf_link, link);
-       event = BPF_CORE_READ(perf_link, perf_file, private_data);
-       return BPF_CORE_READ(event, bpf_cookie);
-}
-
 SEC("iter/task_file")
 int iter(struct bpf_iter__task_file *ctx)
 {
@@ -80,19 +69,9 @@ int iter(struct bpf_iter__task_file *ctx)
        if (file->f_op != fops)
                return 0;
 
-       __builtin_memset(&e, 0, sizeof(e));
        e.pid = task->tgid;
        e.id = get_obj_id(file->private_data, obj_type);
 
-       if (obj_type == BPF_OBJ_LINK) {
-               struct bpf_link *link = (struct bpf_link *) file->private_data;
-
-               if (BPF_CORE_READ(link, type) == BPF_LINK_TYPE_PERF_EVENT) {
-                       e.has_bpf_cookie = true;
-                       e.bpf_cookie = get_bpf_cookie(link);
-               }
-       }
-
        bpf_probe_read_kernel_str(&e.comm, sizeof(e.comm),
                                  task->group_leader->comm);
        bpf_seq_write(ctx->meta->seq, &e, sizeof(e));

[pynem]

@algrebe Thanks for the information! Is there a version that will work with a 5.11 kernel?

[anakryiko]

@qmonnet is there anything we can do to make bpftool a bit less demanding about how up-to-date host kernel on the build machine has to be? Should we just define some of those used constants?

[qmonnet]

Related: https://github.com/libbpf/bpftool/issues/17

My understanding here is that the patch from Alexander would solve the issue upstream. I tried to ping him about it some time ago but didn't get an answer.

I can maybe resubmit this patch this week, once it's merged we can pull it in the bpftool mirror and then in this repo. I'm not aware of any other object, beside struct bpf_perf_link, which should be blocking for bpftool on older kernels at the moment.

How does that sound to you?

[anakryiko]

@qmonnet I think it would be easier and faster to submit bpftool fix separately

[anakryiko]

@pynem can you try again and see if https://github.com/libbpf/libbpf-bootstrap/pull/92 helps?

[dmitris]

tried now and build works fine, thanks! 👍