cross-build (POC)

[frobware]

This PR enables cross-compiling bpfman without using Docker-in-Docker.

Dockerfile:

• Stage 1: Builds bpfman using a Rust base image for the specified target architecture (TARGETARCH).
• Stage 2: Packages the built binaries in a minimal Debian image for runtime.

cross-build.sh:

• Sets up cross-compilation for supported architectures (e.g., aarch64, ppc64le).
• Installs required toolchains and dependencies.
• Configures the environment and builds bpfman for the target using Cargo and the appropriate Rust target.

This approach simplifies cross-compilation by eliminating the need for nested containers.

% uname -m
x86_64

% docker buildx build -f Containerfile.cross-build --build-arg TARGETARCH=aarch64 -t bpfman-aarch64 .
[+] Building 94.0s (18/18) FINISHED                                                                                 docker:default
 => [internal] load build definition from Containerfile.cross-build                                                           0.1s
 => => transferring dockerfile: 760B                                                                                          0.0s
 => [internal] load metadata for docker.io/library/debian:bullseye-slim                                                       0.8s
 => [internal] load metadata for docker.io/library/rust:latest                                                                0.8s
 => [auth] library/debian:pull token for registry-1.docker.io                                                                 0.0s
 => [auth] library/rust:pull token for registry-1.docker.io                                                                   0.0s
 => [internal] load .dockerignore                                                                                             0.0s
 => => transferring context: 65B                                                                                              0.0s
 => [bpfman-build 1/5] FROM docker.io/library/rust:latest@sha256:fcd390e0a3a6bfcf26969861efbe7b864df052aa71a361cf3cd7c5c585b  0.0s
 => [internal] load build context                                                                                             0.4s
 => => transferring context: 287.21MB                                                                                         0.4s
 => [stage-1 1/5] FROM docker.io/library/debian:bullseye-slim@sha256:00558f781b91e90469812bad32002f311ab26ef241b4a1996f66006  0.0s
 => CACHED [bpfman-build 2/5] WORKDIR /usr/src/bpfman                                                                         0.0s
 => [bpfman-build 3/5] COPY . .                                                                                               0.6s
 => [bpfman-build 4/5] RUN ./scripts/cross-build.sh aarch64                                                                  91.5s
 => [bpfman-build 5/5] RUN mkdir bin && cp target/aarch64-unknown-linux-gnu/debug/bpfman* bin                                 0.2s
 => CACHED [stage-1 2/5] WORKDIR /bpfman/bin                                                                                  0.0s
 => CACHED [stage-1 3/5] COPY --from=bpfman-build /usr/src/bpfman/bin/bpfman .                                                0.0s
 => CACHED [stage-1 4/5] COPY --from=bpfman-build /usr/src/bpfman/bin/bpfman-ns .                                             0.0s
 => CACHED [stage-1 5/5] COPY --from=bpfman-build /usr/src/bpfman/bin/bpfman-rpc .                                            0.0s
 => exporting to image                                                                                                        0.0s
 => => exporting layers                                                                                                       0.0s
 => => writing image sha256:601ed06779c80ad183258b3b38f1472732184e9ba1617c331b4f8c7458950d66                                  0.0s
 => => naming to docker.io/library/bpfman-aarch64

#
# Run the container and extract the bpfman-rpc binary.
#
% docker run -d 601ed06779c80ad183258b3b38f1472732184e9ba1617c331b4f8c7458950d66
dd511c6b4ae8985b8550b17da0b887dfc5697cdef5adcaccd218748869b44bab

% docker cp dd511c6b4ae8985b8550b17da0b887dfc5697cdef5adcaccd218748869b44bab:/bpfman/bin/bpfman-rpc .
Successfully copied 287MB to /home/aim/src/github.com/bpfman/bpfman/.

#
# Note that this is an aarch64 binary.
#

% file ./bpfman-rpc
./bpfman-rpc: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=3cb0875fa74059291c792287c89f6301e5cfb914, for GNU/Linux 3.7.0, with debug_info, not stripped

#
# Which we wouldn't expect to run on our x86_64 host.
#
% ./bpfman-rpc
zsh: exec format error: ./bpfman-rpc

#
# Copy this to an arm machine.
#
% scp bpfman-rpc tiwi:
bpfman-rpc

% ssh tiwi

% uname -m
aarch64

% file ./bpfman-rpc
./bpfman-rpc: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=3cb0875fa74059291c792287c89f6301e5cfb914, for GNU/Linux 3.7.0, with debug_info, not stripped

% ldd ./bpfman-rpc
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_GB.utf8)
        linux-vdso.so.1 (0x0000400032a58000)
        libssl.so.3 => /lib/aarch64-linux-gnu/libssl.so.3 (0x0000400032a70000)
        libcrypto.so.3 => /lib/aarch64-linux-gnu/libcrypto.so.3 (0x0000400034e00000)
        libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000400032b30000)
        libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000400034c90000)
        libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000400035260000)
        /lib/ld-linux-aarch64.so.1 (0x0000400032a2e000)

% sudo RUST_LOG=debug ./bpfman-rpc
[DEBUG bpfman_rpc] Log using env_logger
[INFO  bpfman_rpc::serve] Using inactivity timer of 15 seconds
[INFO  bpfman_rpc::serve] Using default Unix socket
[INFO  bpfman_rpc::serve] Listening on /run/bpfman-sock/bpfman.sock
[DEBUG bpfman_rpc::serve] Unix Socket: Received shutdown signal
[DEBUG hyper::server::shutdown] signal received, starting graceful shutdown
[INFO  bpfman_rpc::serve] Shutdown Unix Handler /run/bpfman-sock/bpfman.sock

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 9.05%. Comparing base (f4e9f5f) to head (ba04db1). Report is 16 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff            @@
##            main   #1253      +/-   ##
========================================
- Coverage   9.07%   9.05%   -0.02%     
========================================
  Files         26      24       -2     
  Lines       5953    5950       -3     
========================================
- Hits         540     539       -1     
+ Misses      5413    5411       -2     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[Billy99]

When my PR #1234 merged, we moved Aya sha to the Aya multi-arch support for ppc and s390x. So if you rebase, ppc and s390x will build successfully.

[frobware]

The goal is to pursue a ubi-based build, using native runners for all the supported architectures.