busybox workload started failing with centos stream 9

[anjalirai-intel]

Description of the problem

Workload was successfully running previously, the build started to failing recently on centos stream 9. It is not a Gramine issue and similar build issues are seen with Ubuntu24.04. We have tested latest busybox version on Ubuntu 24.04, but build continued to fail with similar issues.

Previously glibc version was 2.34.105 in centos stream 9 docker image, recently it got updated to 2.34.107 and we suspect this is what is breaking the busybox compilation

Steps to reproduce

ARCH_LIBDIR=/lib64 SGX=1 make all

Expected results

Busybox workload build should be successful and able to run with Gramine

Actual results

Logs: Detailed logs is attached

networking/tc.c: In function ‘cbq_print_opt’:
networking/tc.c:238:27: error: ‘TCA_CBQ_MAX’ undeclared (first use in this function); did you mean ‘TCA_CBS_MAX’?
  238 |         struct rtattr *tb[TCA_CBQ_MAX+1];
      |                           ^~~~~~~~~~~
      |                           TCA_CBS_MAX
networking/tc.c:238:27: note: each undeclared identifier is reported only once for each function it appears in
  CC      libbb/printable_string.o
networking/tc.c:251:16: error: ‘TCA_CBQ_RATE’ undeclared (first use in this function); did you mean ‘TCA_TBF_RATE64’?
  251 |         if (tb[TCA_CBQ_RATE]) {
      |                ^~~~~~~~~~~~
      |                TCA_TBF_RATE64
networking/tc.c:257:16: error: ‘TCA_CBQ_LSSOPT’ undeclared (first use in this function)
  257 |         if (tb[TCA_CBQ_LSSOPT]) {
      |                ^~~~~~~~~~~~~~
networking/tc.c:258:61: error: invalid application of ‘sizeof’ to incomplete type ‘struct tc_cbq_lssopt’
  258 |                 if (RTA_PAYLOAD(tb[TCA_CBQ_LSSOPT]) < sizeof(*lss))
      |                                                             ^
  CC      networking/libiproute/ll_proto.o
  CC      networking/tcpudp_perhost.o
networking/tc.c:263:16: error: ‘TCA_CBQ_WRROPT’ undeclared (first use in this function)
  263 |         if (tb[TCA_CBQ_WRROPT]) {
      |                ^~~~~~~~~~~~~~
networking/tc.c:264:61: error: invalid application of ‘sizeof’ to incomplete type ‘struct tc_cbq_wrropt’
  264 |                 if (RTA_PAYLOAD(tb[TCA_CBQ_WRROPT]) < sizeof(*wrr))

busybox_centos_stream9.log

Do we need to update the documentation for busybox that it does not work on centos stream9 and ubuntu 24.04?

Gramine commit hash

48578a29ab6b0b8a7e75b8bc370e4990f5de91fe

[dimakuv]

This is a bug in Busybox, it is incompatible with the newer Linux kernel versions (and as a result with everything that works closely with newer Linux versions, like Glibc): https://bugs.busybox.net/show_bug.cgi?id=15931

Interestingly, the bug was found in January 2024, but as of now (June 2024) is not yet fixed by Busybox developers -- at least I didn't find any additional info on the bug fix being approved and merged into Busybox.

Instead I notice that at least Gentoo (the Linux OS distribution) has fixed it for its own version of Busybox: https://bugs.gentoo.org/926872

I think the best way is to just wait until Busybox developers fix it and release a new version. Then we can update the Busybox version in our example.

I will now submit a PR that adds a quick documentation about this issue.

[dimakuv]

I added this info to the README of Busybox example, see #1910.

[wuyiao]

I had the same problem

[dimakuv]

Update from 25. September 2024: Busybox support for Ubuntu 24.04 is still pending: https://bugs.busybox.net/show_bug.cgi?id=15931

Let's keep this issue open until Busybox is fixed by its developers, and then we update Busybox version in our CI examples.

[wuyiao]

This is because the kernel version is too new, replacing the Ubuntu24.04 kernel with 6.6.56 will solve the problem