BTF fails to load when run with cilium/ebpf

[coadler]

We have a use case where we're compiling an ebpf program in Rust, but then loading/running it inside Go. The package we're using in Go is https://github.com/cilium/ebpf. But, we seem to be unable to load the elf compiled by redbpf in Go. Below is the error I get:

$ go run -exec sudo main.go
2021/12/02 15:39:53 load and assign: field KprobeOpenat2: program outer_do_sys_openat2: map .rodata: load BTF: invalid argument: magic: 0xeb9f
version: 1
flags: 0x0
hdr_len: 24
type_off: 0
type_len: 412
str_off: 412
str_len: 914
btf_total_size: 1350
[1] PTR *mut c_void type_id=2 Invalid name
exit status 1

I've made an example repo to reproduce this: https://github.com/coadler/redbpf-go. It includes a run.sh script to reproduce the issue. I wasn't too sure where this issue should be made, but the error seems to be coming from the kernel, so I thought it made the most sense here.

[rhdxmr]

Hi @coadler

The bottom line is, unfortunately I couldn't find how to make cilium/ebpf load ELF object file that is compiled by/for RedBPF.

I'm ignorant about cilium/ebpf so I can't tell you that the ELF object file compiled by RedBPF can be loaded by cilium/ebpf successfully in somehow.

But I can tell you that the error message you posted here is the BPF verifier log I am familiar with. As you expected, the BPF verifier log comes from the Linux kernel. And the log says that the type identifier *mut c_void contains invalid characters. Currently the BPF verifier only allows type identifier names that are compatible with C language. As you know *mut c_void is not a valid thing in C language. So it is rejected by the verifier.

Actually RedBPF also suffered this issue but it avoids the error by substituting all invalid characters with underscores just before it loads BTF into the Linux kernel. But I guess cilium/ebpf does not conduct that kind of process.

---

I had never used go language before but I cloned your repo and installed golang to test your program.

I could resolve the error you posted by wiping out the problematic BTF section from the ELF relocatable file. You can see the command I used:

diff --git a/run.sh b/run.sh
index a90e514..08b5b57 100755
--- a/run.sh
+++ b/run.sh
@@ -5,6 +5,7 @@ set -euo pipefail
 pushd probes
     echo Building elf..
     cargo bpf build --target-dir=../target
+    llvm-strip --remove-section .BTF --remove-section .BTF.ext --no-strip-all ../target/bpf/programs/openmonitor/openmonitor.elf
 popd

 pushd cmd/ebpf

But this leads to another problem that I can't dive into. Perhaps it originates from cilium/ebpf:

2021/12/03 12:10:20 load spec: load data sections: data sections require BTF,
make sure all consts are marked as static

It seems that the ELF object file compiled by RedBPF is not the one that cilium/ebpf expects. Both RedBPF and cilium/ebpf handle ELF object files but the formats of the content inside them are probably different.

Inside ELF object file, there are several ELF sections. Each section has their names and data. RedBPF has its own rules for section names and specific format for data. And cilium/ebpf also has its own rules and formats.

So we can't assume the ELF object files are inter-operable between them. RedBPF runtime can not handle the ELF object file that is compiled for cilium/ebpf and vice versa.