How distributable are XDP userland binaries and kernel bpf binaries?

[simonhf]

The XDP tutorial setup dependencies [1] say:

For now, this tutorial depends on kernel headers package provided by your distro. We may choose to shadow some of these later.

Let's say I create an example_kern.o binary on my Linux box. Will it just work on all other Linux boxes with the same kernel or better? Or what are the limitations?

And let's say I create a statically linked example_user binary with Alpine Linux [2] using musl [3]. Will it just work on all other Linux boxes with the same CPU manufacturer and same kernel or better? Or what are the limitations?

Which issues might I run into? And do you have any advice for creating "run anywhere" distributable XDP userland binaries and kernel bpf binaries? And does anybody know of existing projects distributing XDP userland binaries and kernel bpf binaries?

Thanks!

[1] https://github.com/xdp-project/xdp-tutorial/blob/master/setup_dependencies.org [2] https://alpinelinux.org/ [3] https://www.musl-libc.org/

[tohojo]

Simon Hardy-Francis [email protected] writes:

The XDP tutorial setup dependencies [1] say:

For now, this tutorial depends on kernel headers package provided by your distro. We may choose to shadow some of these later.

Let's say I create an example_kern.o binary on my Linux box. Will it just work on all other Linux boxes with the same kernel or better? Or what are the limitations?

Generally, the interface exposed to a BPF program is considered kernel UAPI, so carries the same backwards compatibility guarantees as any other kernel UAPI. I.e., you BPF program should keep working on newer versions of the kernel.

For older versions things get a little more complicated, as there may be verifier features and helpers missing on those older kernels. However, to a certain extent it is possible to write a BPF program where the same binary can be loaded across different kernel versions, but you will have to incorporate some compatibility handling in the BPF program itself, which will be resolved at load time. This is called BPF CO-RE (Compile Once - Run Everywhere). See this post for details: https://facebookmicrosites.github.io/bpf/blog/2020/02/19/bpf-portability-and-co-re.html

And let's say I create a statically linked example_user binary with Alpine Linux [2] using musl [3]. Will it just work on all other Linux boxes with the same CPU manufacturer and same kernel or better? Or what are the limitations?

Nothing special about userland binaries for BPF; the bpf() system call is just a normal system call. So yeah, that should work. If it doesn't, that's a bug; those do happen of course, but if you report them there should be no issue having them fixed.

Which issues might I run into? And do you have any advice for creating "run anywhere" distributable XDP userland binaries and kernel bpf binaries?

I think the main thing that's missing is an explicit way to query the kernel for which XDP features are supported by a specific driver. So absent this, I think the main issue you will run into is that your program will work on some combinations of network devices and not on others. And the failure mode is not great (silently dropped packets). We do want to add support for querying the kernel about support, see: https://xdp-project.net/#XDP-feature-flags

However, until that lands I fear you may have to maintain your own compatibility list of which drivers are support on which kernel versions and react to that in your application (or at least document it).