Building with newer compilers to not miss interesting optimizations

[illwieckz]

We build our releases on an old distribution (Debian Buster) because that's an easy way to produce binaries compatibles with older distros. Especially, we do that to link against an old libc and then don't distribute binaries that would require a too-new libc.

The drawback is that we then use the old compiler of such distribution, we may then miss some optimizations.

Here is an example with gcc on Debian Buster, compiling libjpeg-turbo for arm64: NEON SIMD optimizations are not supported and then unused:

$ ./build.sh linux-arm64-default jpeg
-- The C compiler identification is GNU 8.3.0
…
-- Compiler flags =  -fPIC -march=armv8-a -O3 -DNDEBUG
…
-- Performing Test HAVE_VLD1_S16_X3
-- Performing Test HAVE_VLD1_S16_X3 - Failed
-- Performing Test HAVE_VLD1_U16_X2
-- Performing Test HAVE_VLD1_U16_X2 - Success
-- Performing Test HAVE_VLD1Q_U8_X4
-- Performing Test HAVE_VLD1Q_U8_X4 - Failed
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/aarch64-linux-gnu-gcc
-- Use partial Neon SIMD intrinsics implementation (NEON_INTRINSICS = OFF)
-- CMAKE_ASM_FLAGS =  -fPIC -march=armv8-a  -O3 -DNDEBUG
-- SIMD extensions: arm64 (WITH_SIMD = 1)

Here is an example with gcc on more recent Ubuntu Kinetic 22.10, compiling libjpeg-turbo for arm64: NEON SIMD optimizations are supported and then used:

$ ./build.sh linux-arm64-default jpeg
-- The C compiler identification is GNU 12.2.0
…
-- Compiler flags =  -fPIC -march=armv8-a -O3 -DNDEBUG
…
-- Performing Test HAVE_VLD1_S16_X3
-- Performing Test HAVE_VLD1_S16_X3 - Success
-- Performing Test HAVE_VLD1_U16_X2
-- Performing Test HAVE_VLD1_U16_X2 - Success
-- Performing Test HAVE_VLD1Q_U8_X4
-- Performing Test HAVE_VLD1Q_U8_X4 - Success
-- Use full Neon SIMD intrinsics implementation (NEON_INTRINSICS = ON)
-- SIMD extensions: arm64 (WITH_SIMD = 1)
-- FLOATTEST = fp-contract

One solution would be to use a newer compiler on an old distribution using an old libc. Unfortunately GCC doesn't provide updated builds for Debian (unlike what LLVM/Clang does) and we use GCC for everything that isn't macOS. One may use crosstool-NG to build a newer GCC.

Another solution would be to use (and then build) an older libc on a newer distribution providing a newer compiler.

[slipher]

I think it's probably safe to build our engine with a new compiler. In most cases a dependency on newer libc only appears if you actually use a new function that's not supported in old ones. If I'm wrong, validate-release will inform us thanks to its check on libc and libcxx versions.

Building external_deps could be dangerous though. The config scripts there might enable stuff that depends on a new libc if they detect a newer one.

[illwieckz]

I found this that is said to help to build a software by targetting a specific libc version, but there is only support for x86/x86_64, has not be updated in years, and is only about libc (not libc++):

• https://github.com/wheybags/glibc_version_header

[illwieckz]

@slipher, what would be the possible drawbacks of building against an old libc?

The crosstool-NG tool can select which glibc build when building a compiler, and it lists a long list of possible glibc to build (and some very old ones), so I wonder what would be the inconvenient of picking an old one.