Corrupt distribution artifacts due to old carray files

[mkroening]

I found this issue to be the root cause of our bootloader (hermit-os/loader) not booting anymore when upgrading to the OpenSBI 1.6 release artifacts (we use FW_JUMP with PLATFORM=generic and XLEN=64).

I bisected ./scripts/create-binary-archive.sh creating a faulty firmware to be caused by fd9e8b17ed5017d2914f5fc7eb42ff24ff5c097b. Cleanly building just the firmware that we need with make PLATFORM=generic XLEN=64 does not have this issue.

This seems to be caused by build/lib/sbi/sbi_ecall_exts.carray.c being generated as empty when building libsbi, which is not regenerated when building the firmware, for which it should not be empty, apparently.

This can be reproduced as follows:

make XLEN=64 install_libsbi
rm build/lib/sbi/sbi_ecall_exts.carray.c
rm build/lib/sbi/sbi_ecall_exts.carray.o
make PLATFORM=generic XLEN=64

When removing the generated carray files in between the builds, the carray populated and the firmware boots correctly. When not removing the files, the carray is not populated and is not properly regenerated for the firmware build.

How should we solve this? Should we differentiate between carray files generated for different targets or their configuration?

[avpatel]

Can you try to following changes and see if it works ?

diff --git a/scripts/create-binary-archive.sh b/scripts/create-binary-archive.sh
index 9d56f2f4..a07a57b5 100755
--- a/scripts/create-binary-archive.sh
+++ b/scripts/create-binary-archive.sh
@@ -117,6 +117,7 @@ build_opensbi() {
        # Build and install generic library
        echo "Build and install generic library XLEN=${BUILD_RISCV_XLEN}"
        echo ""
+       rm -rf "${BUILD_OUTPUT_PATH}/${BUILD_NAME}"
        make -C "${BUILD_OPENSBI_SOURCE_PATH}" O="${BUILD_OUTPUT_PATH}/${BUILD_NAME}" I="${BUILD_OUTPUT_PATH}/${BUILD_ARCHIVE_NAME}" PLATFORM_RISCV_XLEN="${BUILD_RISCV_XLEN}" install_libsbi -j "${BUILD_NUM_THREADS}"
        echo ""
 
@@ -125,6 +126,7 @@ build_opensbi() {
        do
                echo "Build and install PLATFORM=${BUILD_PLATFORM_SUBDIR[${INDEX}]} XLEN=${BUILD_RISCV_XLEN}"
                echo ""
+               rm -rf "${BUILD_OUTPUT_PATH}/${BUILD_NAME}"
                make -C "${BUILD_OPENSBI_SOURCE_PATH}" O="${BUILD_OUTPUT_PATH}/${BUILD_NAME}" I="${BUILD_OUTPUT_PATH}/${BUILD_ARCHIVE_NAME}" PLATFORM="${BUILD_PLATFORM_SUBDIR[${INDEX}]}" PLATFORM_RISCV_XLEN="${BUILD_RISCV_XLEN}" install_libplatsbi install_firmwares -j "${BUILD_NUM_THREADS}"
                echo ""
        done

[mkroening]

Thanks! Yes, that works. 👌

[SiFiveHolland]

This is a deeper problem with the OpenSBI build system. The underlying issue that build artifacts are shared between platforms is fixed by this patch: https://patchwork.ozlabs.org/project/opensbi/patch/[email protected]/

[mkroening]

Ah, okay. Would it make sense to publish new artifacts for 1.6 (or release 1.6.1) using @avpatel's patch before fixing this properly for the next release?

[avpatel]

This is a deeper problem with the OpenSBI build system. The underlying issue that build artifacts are shared between platforms is fixed by this patch: https://patchwork.ozlabs.org/project/opensbi/patch/[email protected]/

I agree with you but the change which you pointed is incomplete. We also need to update scripts and documentation as well so that users don't expect generic libsbi.a from OpenSBI.

[avpatel]

I have updated the release tarball for v1.6 (https://github.com/riscv-software-src/opensbi/releases/tag/v1.6) Please check at your end.

[mkroening]

Yes, that works! Thank you so much! :)