sel4bench
sel4bench copied to clipboard
CI actions should build with clang also
CI actions should build with clang also besides just gcc and benchmaks should run with both binaries. Building with clang is expected to improve code quality, because clang raises more warnings. Large discrepancies in the benchmark results can indicate that the benchmark either has a strong compiler dependency or its implementation has issues.
Building with clang also seems rather trivial, running both binaries will be more work.
That does sound reasonable. @kent-mcleod any concerns on adding a build and run with clang for all benchmarks? It'll take more machine time, but it could be informative.
Then again, the x86 runs each take 50 min, and if we want comparable numbers they need to run on the same machine, so we would be blowing the time out to 100 min.
If the concern is about hardware resources, it could be a topic for the budget planning.
If we want gcc/clang to be comparable, should it not be the exact same machine? In that case it doesn't matter if we buy more. We do have more x86 boards, but I'm not sure they are the same configurations. @wom-bat might be able to confirm. x86 machines should be easy to add, I think, if that helps.
Another way to look at this would be to see if all x86 benchmarks are actually needed/informative. What currently exists is what has accumulated and not necessarily designed to be the optimal set for CI.