Xilinx-CNS
onload compilation fails on Ubuntu 24.04 ARM64
Hi, wonder if anyone has encountered the issue or has an idea off the top on his/her head for the cause
Encountered issue
Following compilation instructions:
export PATH="$PWD/scripts:$PATH"
mmakebuildtree --driver
mmakebuildtree --user
make -C "$(mmaketool --toppath)/build/$(mmaketool --driverbuild)"
On Ubuntu 24.04, raises the following errors:
onload/src/driver/linux_resource/syscall_aarch64.c:47:20: error: redefinition of ‘aarch64_insn_is_bti’
47 | static inline bool aarch64_insn_is_bti(u32 code)
...
onload/src/driver/linux_onload/onload_kernel_compat.h:65:19: error: redefinition of ‘is_compat_task’
65 | static inline int is_compat_task(void)
applying this small fix:
diff --git a/src/driver/linux_onload/onload_kernel_compat.h b/src/driver/linux_onload/onload_kernel_compat.h
index 3ca63c79b914..9821f21611d8 100644
--- a/src/driver/linux_onload/onload_kernel_compat.h
+++ b/src/driver/linux_onload/onload_kernel_compat.h
@@ -61,7 +61,7 @@ ci_call_usermodehelper(char *path, char **argv, char **envp, int wait);
#endif
/* is_compat_task() was removed for x86 in linux-4.6 */
-#ifdef EFRM_NEED_IS_COMPAT_TASK
+#if defined(EFRM_NEED_IS_COMPAT_TASK) && !defined(CONFIG_ARM64)
static inline int is_compat_task(void)
{
#if !defined(CONFIG_COMPAT)
diff --git a/src/driver/linux_resource/syscall_aarch64.c b/src/driver/linux_resource/syscall_aarch64.c
index 46319cad41e3..70a070a9e771 100644
--- a/src/driver/linux_resource/syscall_aarch64.c
+++ b/src/driver/linux_resource/syscall_aarch64.c
@@ -44,12 +44,12 @@ static typeof(aarch64_insn_adrp_get_offset) *ci_aarch64_insn_adrp_get_offset;
/* Linux does not have any function to check for 'bti' instruction. So we
* define it by ourselves. */
-static inline bool aarch64_insn_is_bti(u32 code)
-{
- u32 mask = 0xFFFFFF3F;
- u32 val = 0xD503241F;
- return (code & mask) == val;
-}
+// static inline bool aarch64_insn_is_bti(u32 code)
+// {
+// u32 mask = 0xFFFFFF3F;
+// u32 val = 0xD503241F;
+// return (code & mask) == val;
+// }
/* Depending on the kernel version, locating the syscall table may be more
* or less straigthforward.
does manage to compile the driver.
Nevertheless, when trying to compile the userspace component with:
make -C "$(mmaketool --toppath)/build/$(mmaketool --userbuild)"
The compilation starts failing with a more serious issue:
onload/src/lib/ciul/shrub_server.c:656:3: error: implicit declaration of function ‘ci_bit_for_each_set’ [-Werror=implicit-function-declaration]
656 | ci_bit_for_each_set(ix, (const ci_bits*)server->vi->efct_rxqs.active_qs,
which makes sense considering that the macro ci_bit_for_each_set is only defined in src/include/ci/tools/platform/gcc_x86.h and not in src/include/ci/tools/platform/gcc_aarch64.h.
Tried digging into the build system, but I couldn't find how to disable the unit-testing compilation (if this is even possible), not sure if it's the only part which doesn't seem to compile on ARM.
System information
I'm running Ubuntu 24.04 AWS server version with a vanilla kernel 6.9 (compiled with the same configuration as the distributed one, but with MTD module added).
$ gcc --version
gcc (Ubuntu 13.2.0-23ubuntu4) 13.2.0
Different driver component compilation with Amazon Linux 2023
Tried to also run the same compilation test on a distribution onload used to compile on (though can't remember which version \=).
The same compilation error for ci_bit_for_each_set is encountered there as well, but interestingly enough the driver part is able to compile without any modifications.
Might be a compatibility thing. Amazon Linux 2023 runs kernel 6.1.112 and gcc (GCC) 11.4.1. I ran a dry run on both distros and compared the files which are being compiled. The same 308 files are compiled on both.
In any rate since it's not a supported distribution according to the main readme, I'm mostly concerned with the Ubuntu compilation error.
Hey, I am also trying to compile this in my RK3588 chip. Did you managed to get this working?
Hi, I have a tree based on current git main that compiles for ARM Ubuntu 24.04, if anyone is interested can make available. You need to ensure you are using s kernel with 4k pagesize:
getconf PAGESIZE
4096
However getting this message when trying to load the kernel modules
# onload_tool reload
onload_tool: /sbin/modprobe -r sfc
onload_tool: /sbin/modprobe sfc
onload_tool: /sbin/modprobe onload
modprobe: ERROR: could not insert 'onload': Unknown symbol in module, or unknown parameter (see dmesg)
onload_tool: FAILED: /sbin/modprobe onload
dmesg output:
....
[17084.109089] Solarflare NET driver unloading
[17084.265130] Solarflare NET driver v6.1.0.1008
[17084.298673] [sfc efrm] find_syscall_table_via_vbar:388: expected adrp instruction at ffffb80c9a8dd98c, found d5385214
[17084.298689] [sfc efrm] init_sfc_resource: ERROR: failed to find syscall table
Anyone any ideas?
Kernel version :
Linux localhost 6.8.0-60-generic #63-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 15 18:51:58 UTC 2025 aarch64 aarch64 aarch64 GNU/Linux
Finally had time to look at this and put the code up. The issue "find_syscall_table_via_vbar" took a little time, and I wouldn't describe my solution as "good". In the end, I created a helper script in user space to pass the memory address ie modprobe "sfc_resource syscall_table_addr=0xffffb80c9a910a38“.
https://github.com/IntelligentCompute/onload-arm
Everything now loads and initialises.
onload_tool reload
onload_tool: /sbin/modprobe -r onload
onload_tool: /sbin/modprobe -r sfc_resource
onload_tool: /sbin/modprobe -r sfc
onload_tool: Detected ARM64 kernel 6.8+, using ARM64 helper for module loading
onload_tool: /usr/sbin/onload_arm64_helper
Loading sfc driver...
Loading sfc_resource with syscall_table_addr=0xffffb80c9a910a38...
Loading sfc_char...
ARM64 helper: Ready for onload module loading
onload_tool: /sbin/modprobe --ignore-install onload