eBPF CO-RE improvements: Refactoring tracee.bpf.c

[simar7]

Motivation: When I was trying to write a simple unit test for functions in tracee.bpf.c I found it extremely challenging to set up all the necessary headers in place for me to write a user space C program which could call the functions tracee.bpf.c has.

What I would like to see: We can split up tracee.bpf.c whichever best way, into smaller manageable chunks of code. This way the smaller units could have tests written for without the hassle to include all dependencies at once.

Example: I took one small function and extracted it out to a file of its own. Then I created a user space program to call this function directly. You can see the example here: https://github.com/simar7/ctesting/blob/master/tracee.bpf.c (toy program) and https://github.com/simar7/ctesting/blob/master/test_tracee.bpf.c (test)

Thoughts? Or any other ideas how we can test the C code?

[simar7]

One additional way @grantseltzer brought up was to use libbpfgo wrapper to call underlying C code. While this makes testing a whole lot simpler it does have an overhead of creating an object file and loading prior to running tests. At which point they wouldn't be unit tests anymore - but maybe that's an acceptable trade-off here?

[itaysk]

thanks for starting this discussion!

1. I'm all for refactoring the C code if it's practical (we need to be careful when messing with the bpf verifier)
2. the example function you used is a rare example where the code is pure C, but in most cases there will be calls out to kernel code / bpf helpers etc, which we will need to mock.
3. instead of using libbpfgo, we can just use cgo

[yanivagman]

Agree with @itaysk - good comments.

@simar7 - I'm not sure what is the overhead you are talking about. Anyways you will have to create (compile) an object file from the C code, won't you? We had a good discussion (open issue) about how we should perform the tests in general (including unit tests), whose scope was changed as it was too wide: #253

[simar7]

I keep meaning to reply on this discussion but keep forgetting – I read through #253 there are some good ideas raised.

After thinking about it some more I wonder if there's value in writing pure unit tests (mocking external dependencies like kernel code etc.) here. This might be one of those situations where writing an integration test is not only easier but also closer to reality (akin to networking tests).

I like the selftest approach @grantseltzer has taken writing the tests for the new ring buffer stuff https://github.com/aquasecurity/tracee/blob/6c8d7f2d76eb2daa743e418d534db11dab05ab44/libbpfgo/selftests/5.8.15/ringbuffers/main.go – could we do something similar with the rest of tracee.bpf.c?

[itaysk]

comments from dup issue:

currently tracee.bpf.c is one huge file that's not so easy to comprehend, we should split it into smaller files. probably we want to include the files into one, so it's easier to compile. first half of the file is mostly helpers, which can be extracted possibly split every ebpf program into a file (TBD directory structure).

related: #1160