continuous-integration icon indicating copy to clipboard operation
continuous-integration copied to clipboard

Published 20 hours ago • verified publisher icon bazelbuild

Make `C.UTF-8` locale available on CentOS 7

Open fmeum opened this issue 1 year ago • 5 comments

CentOS 7 is EoL and lacks a C.UTF-8 locale. Simulate it while CI hasn't been updated yet to unblock migrating Bazel to consistently use a UTF-8 locale.

fmeum avatar Oct 17 '24 14:10 fmeum

@meteorcloudy This is needed to unblock https://github.com/bazelbuild/bazel/pull/24021.

fmeum avatar Oct 17 '24 15:10 fmeum

This is a bit tricky. I tried to rebuild all docker images two days ago, and the centos7 image is already not buildable due to https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm being 404.

We should just switch to a successor of centos7, maybe rocky linux 8.

What matters is the glibc version for the release build, we want a low glibc version to make sure Bazel can work on older Linux distribution. Currently we have:

  • centos7 (EOL): ldd (GNU libc) 2.17
  • debain10 (EOL): ldd (Debian GLIBC 2.28-10+deb10u3) 2.28
  • ubuntu1804 (EOL): ldd (Ubuntu GLIBC 2.27-3ubuntu1.6) 2.27
  • debian11: ldd (Debian GLIBC 2.31-13+deb11u11) 2.31
  • ubuntu2004: ldd (GNU libc) 2.31
  • rockylinux:8 (a popular successor to centos): ldd (GNU libc) 2.28

/cc @fweikert

@fmeum Can you just skip broken tests on centos7 for now? I'll find some time to properly migrate it to a new platform.

meteorcloudy avatar Oct 18 '24 08:10 meteorcloudy

Will do!

If you want to avoid having to maintain an old distro in CI, but want to keep a low minimum glibc version, you could also use a hermetic toolchain that compiles against a sysroot (e.g. one from Chromium).

fmeum avatar Oct 18 '24 08:10 fmeum

Interesting! Can you give me some pointer to the hermetic toolchain?

meteorcloudy avatar Oct 18 '24 09:10 meteorcloudy

It's mostly just this (the bug mentioned in the comment below has been fixed since): https://github.com/CodeIntelligenceTesting/jazzer/blob/aa376ca3c0255b9cf04e1fc79fcf1fe5c63b2376/MODULE.bazel#L397

The only caveat is that the official LLVM releases are extremely bloated and slow to extract even on fast machines. There are multiple options around this:

  1. Use toolchains_musl to link against musl instead. Super small toolchain and no need to maintain a sysroot, but musl may regress performance in the native code parts. If you have a way to rule this out with benchmarks, it would be the easiest to maintain in the long run.
  2. Use a much smaller LLVM distribution: https://github.com/dzbarsky/static-clang

fmeum avatar Oct 18 '24 09:10 fmeum