bpftime icon indicating copy to clipboard operation
bpftime copied to clipboard

[BUG] `bpftime load` always enforce the userland ebpf program to use bpftime's own libbpf

Open agentzh opened this issue 1 year ago • 1 comments

My ebpf userland program is dynamically linked with my patched version of libbpf. The program can find the correct libbpf via RPATH. But bpftime load or the LD_PRELOAD way always forces my userland program to use bpftime's own libbpf, which is actually incompatible. How to work around this issue? I think by design, bpftime supports transparent operations of the original user ebpf programs (including userland) and user's own libbpf, right?

agentzh avatar Jan 28 '24 05:01 agentzh

OK, after some more digging, it seems like my libbpf.so is also loaded, but the statically linked libbpf in bpftime takes priority. So if libbpf API functions exist in bpftime's libbpf, this version of libbpf is used; otherwise, it falls to my libbpf.so. So the actual result is even more bizarre: sometimes it uses bpftime's libbpf and sometimes it uses mine. This is so confusing and hard to debug...

Ideally, it should always use the original userland ebpf program's libbpf. bpftime's libbpf should only be used for kernel interoperability. In fact, for my use case, I never use kernel interoperability, so the easier fix might be just to implement the feature in #175.

agentzh avatar Jan 28 '24 06:01 agentzh

A solution:

  • Set symbol visibility to hidden for all symbols in agent and syscall server. A hidden symbol appeared in LD_PRELOAD library won't override symbol in the injected executable
  • More precisely, we may only set the visibility of symbols from libbpf to hidden, others left default. This may require a custom ld script. I'll investigate into it

Officeyutong avatar Jun 13 '24 15:06 Officeyutong