CI: packages: Add github CI job to build all packages

[hauke]

This will build OpenWrt for MIPS malta BE with all core packages activated. It is triggered when something changes in the build system or when a package definition is changed. This task probably needs multiple hours to execute, but I hope that it will find build problems in pull requests early.

This is based on the .github/workflows/packages.yml workflow.

Signed-off-by: Hauke Mehrtens [email protected]

[aparcar]

Why that arch specifically?

[hauke]

I did this only for mips 32 be because this will take some time to build. I think we do not have enough resources to build it for all targets. I am thinking about to add an other with experimental toolchain like gcc 12 and so on.

[hauke]

The kernel build takes 22m 50s here, for the all targets build it is about 10 minutes.This workflow will build all core packages and kernel modules and it also found a problem.

It failed because of this problem, that is getting fixed in #10396, I do not know why our build bots do not fail on this.

Packaged contents of /__w/openwrt/openwrt/openwrt/build_dir/target-mips-openwrt-linux-musl_musl/linux-malta_be/packages/ipkg-mips_24kc/kmod-phylink into /__w/openwrt/openwrt/openwrt/bin/targets/malta/be/packages/kmod-phylink_5.10.135-1_mips_24kc.ipk
Package kmod-mdio-devres is missing dependencies for the following libraries:
of_mdio.ko
make[3]: *** [modules/netdevices.mk:155: /__w/openwrt/openwrt/openwrt/bin/targets/malta/be/packages/kmod-mdio-devres_5.10.135-1_mips_24kc.ipk] Error 1
time: package/kernel/linux/compile#25.39#6.55#35.41

[ynezz]

I am thinking about to add an other with experimental toolchain like gcc 12 and so on.

Having one BE and one LE target is certainly a good idea. I would vote for x86/64 as that 2nd one due to KVM (much faster in QEMU), familiarity etc.

[hauke]

I am getting the following compile errors: trace-cmd-v3.1.2 on malta be:

  COMPILE FPIC           trace-timesync-ptp.o
  COMPILE FPIC           trace-timesync-kvm.o
trace-timesync-kvm.c: In function 'kvm_open_vcpu_dir':
trace-timesync-kvm.c:221:19: error: 'PATH_MAX' undeclared (first use in this function)
  221 |         char path[PATH_MAX];
      |                   ^~~~~~~~
trace-timesync-kvm.c:221:19: note: each undeclared identifier is reported only once for each function it appears in
trace-timesync-kvm.c: In function 'kvm_open_debug_files':
trace-timesync-kvm.c:278:19: error: 'PATH_MAX' undeclared (first use in this function)
  278 |         char path[PATH_MAX];
      |                   ^~~~~~~~
make[5]: *** [Makefile:72: /__w/openwrt/openwrt/openwrt/build_dir/target-mips-openwrt-linux-musl_musl/trace-cmd-v3.1.2/lib/trace-cmd/trace-timesync-kvm.o] Error 1
make[4]: *** [Makefile:405: /__w/openwrt/openwrt/openwrt/build_dir/target-mips-openwrt-linux-musl_musl/trace-cmd-v3.1.2/lib/trace-cmd/libtracecmd.a] Error 2
make[4]: Leaving directory '/__w/openwrt/openwrt/openwrt/build_dir/target-mips-openwrt-linux-musl_musl/trace-cmd-v3.1.2'
make[3]: *** [Makefile:50: /__w/openwrt/openwrt/openwrt/build_dir/target-mips-openwrt-linux-musl_musl/trace-cmd-v3.1.2/.built] Error 2
time: package/devel/trace-cmd/compile#3.41#0.35#4.12

iucode-tool-2.3.1 on x86_64:

x86_64-openwrt-linux-musl-gcc -DHAVE_CONFIG_H -I.   -I/__w/openwrt/openwrt/openwrt/openwrt-toolchain-x86-64_gcc-11.3.0_musl.Linux-x86_64/toolchain-x86_64_gcc-11.3.0_musl/usr/include -I/__w/openwrt/openwrt/openwrt/openwrt-toolchain-x86-64_gcc-11.3.0_musl.Linux-x86_64/toolchain-x86_64_gcc-11.3.0_musl/include   -Os -pipe -fno-caller-saves -fno-plt  -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro  -MT iucode_tool.o -MD -MP -MF .deps/iucode_tool.Tpo -c -o iucode_tool.o iucode_tool.c
iucode_tool.c: In function 'load_intel_microcode_bin':
iucode_tool.c:1088:45: error: 'SSIZE_MAX' undeclared (first use in this function); did you mean 'SIZE_MAX'?
 1088 |                           (mcb_space_left < SSIZE_MAX) ? mcb_space_left : SSIZE_MAX);
      |                                             ^~~~~~~~~
      |                                             SIZE_MAX
iucode_tool.c:1088:45: note: each undeclared identifier is reported only once for each function it appears in
iucode_tool.c: In function 'load_intel_microcode':
iucode_tool.c:1268:20: error: 'PATH_MAX' undeclared (first use in this function)
 1268 |         char fnbuf[PATH_MAX];
      |                    ^~~~~~~~
iucode_tool.c: In function 'ssizemax_clamp':
iucode_tool.c:1429:21: error: 'SSIZE_MAX' undeclared (first use in this function); did you mean 'SIZE_MAX'?
 1429 |         return (s < SSIZE_MAX) ? (size_t)s : SSIZE_MAX;
      |                     ^~~~~~~~~
      |                     SIZE_MAX
In file included from iucode_tool.c:29:
iucode_tool.c: In function 'write_cpio_header':
iucode_tool.c:1731:22: error: 'SSIZE_MAX' undeclared (first use in this function); did you mean 'SIZE_MAX'?
 1731 |         assert(pos < SSIZE_MAX);
      |                      ^~~~~~~~~
iucode_tool.c: In function 'xx_check_cpuid_devs':
iucode_tool.c:2767:27: error: 'PATH_MAX' undeclared (first use in this function)
 2767 |         char cpuid_device[PATH_MAX];
      |                           ^~~~~~~~
make[5]: *** [Makefile:440: iucode_tool.o] Error 1
make[5]: Leaving directory '/__w/openwrt/openwrt/openwrt/build_dir/target-x86_64-openwrt-linux-musl_musl/iucode-tool-2.3.1'
make[4]: *** [Makefile:329: all] Error 2
make[4]: Leaving directory '/__w/openwrt/openwrt/openwrt/build_dir/target-x86_64-openwrt-linux-musl_musl/iucode-tool-2.3.1'
make[3]: *** [Makefile:65: /__w/openwrt/openwrt/openwrt/build_dir/target-x86_64-openwrt-linux-musl_musl/iucode-tool-2.3.1/.built] Error 2
time: package/system/iucode-tool/compile#1.88#0.52#2.48

I have to look more closely into these problems.

[hauke]

trace-cmd-v3.1.2 and iucode-tool-2.3.1 are failing to compile because we do not add this: -I$(TOOLCHAIN_DIR)/include/fortify when we use an external musl toolchain.For internal toolchain it is done here: https://github.com/openwrt/openwrt/blob/master/rules.mk#L190

I will check if this means that fortify source is not working when using external musl toolchains.

[Ansuel]

(btw a bit ot but we lack workflow to test toolchain changes)

[stintel]

Having one BE and one LE target is certainly a good idea. I would vote for x86/64 as that 2nd one due to KVM (much faster in QEMU), familiarity etc.

Also building for x86/64 on x86/64 is extra sensitive for host lib leakage iirc, so it's definitely a good candidate imo.

[stintel]

Is there an easy way to also enable all CONFIG_KERNEL_* symbols? There's a lot of breakage in some of them when we're adding a new kernel.

[Ansuel]

@hauke considering we already have test for tools why not use the tools container also for this?

[hauke]

Yes I will try to use the pre build tools. unetd is still not building without the LLVM toolchain.

 make[3] -C package/network/services/unetd compile
    ERROR: package/network/services/unetd failed to build.
make -r world: build failed. Please re-run make with -j1 V=s or V=sc for a higher verbosity level to see what's going on
make: *** [/__w/openwrt/openwrt/openwrt/include/toplevel.mk:231: world] Error 1

[Ansuel]

I'm probably blind but the tools is not getting compiled? Or if it should be part of the toolchain are we sure we are shipping external toolchain with them?

time: target/linux/prereq#0.09#0.04#0.12
 make[1] tools/install
 make[2] -C tools/flock compile
 make[2] -C tools/xz compile
 make[2] -C tools/sed compile
 make[2] -C tools/tar compile
 make[2] -C tools/patch compile
 make[2] -C tools/m4 compile
 make[2] -C tools/autoconf-archive compile
 make[2] -C tools/ninja compile
 make[2] -C tools/expat compile
 make[2] -C tools/zlib compile
 make[2] -C tools/cpio compile
 make[2] -C tools/lzma compile
 make[2] -C tools/make-ext4fs compile
 make[2] -C tools/mtools compile
 make[2] -C tools/patch-image compile
 make[2] -C tools/squashfskit4 compile
 make[2] -C tools/sstrip compile
 make[2] -C tools/zip compile
 make[2] -C tools/elfutils compile
 make[2] -C tools/autoconf compile
 make[2] -C tools/meson compile
 make[2] -C tools/missing-macros compile
 make[2] -C tools/zstd compile
 make[2] -C tools/pkgconf compile
 make[2] -C tools/automake compile
 make[2] -C tools/libressl compile
 make[2] -C tools/dosfstools compile
 make[2] -C tools/libtool compile
 make[2] -C tools/cmake compile
 make[2] -C tools/flex compile
 make[2] -C tools/e2fsprogs compile
 make[2] -C tools/fakeroot compile
 make[2] -C tools/gengetopt compile
 make[2] -C tools/mklibs compile
 make[2] -C tools/mtd-utils compile
 make[2] -C tools/patchelf compile
 make[2] -C tools/bison compile
 make[2] -C tools/bc compile
 make[2] -C tools/findutils compile
 make[2] -C tools/mkimage compile
 make[2] -C tools/padjffs2 compile
 make[2] -C tools/quilt compile
 make[2] -C tools/b43-tools compile
 make[2] -C tools/firmware-utils compile

[hauke]

I will try to use prebuild tools like done in 5d09118f8e60fa151e03916f255f5511e197af68 and c27b43956407f3adc3cc2693792acd6b40a01877

[Ansuel]

@hauke remember to set the correct permission

[hauke]

@stintel We do not have a option to build all CONFIG_KERNEL_* options, you could add it.

I am using now the tools container like it is done in the kernel.yml file. I syned the code again with the kernel.yml, but did not add the ccache. I would like to detect problems like a gdb build failure after the readĺine upgrade, are they still detected with ccache?

I also updated the permissions to match kernel.yml.