grafana
GLIBC version mismatch in Promtail binary
Describe the bug When attempting to run Promtail, the following error occurs: /usr/bin/promtail: /lib/x86_64-linux-gnu/libc.so.6: version GLIBC_2.32' not found (required by /usr/bin/promtail) /usr/bin/promtail: /lib/x86_64-linux-gnu/libc.so.6: version GLIBC_2.34' not found (required by /usr/bin/promtail)
To Reproduce Install and run promtail 3.1.0 from apt.grafana.com using instructions from https://apt.grafana.com
Expected behavior Promtail should run without GLIBC version errors.
Environment:
- local install on Debian 11 (bullseye)
Screenshots, Promtail config, or terminal output /usr/bin/promtail: /lib/x86_64-linux-gnu/libc.so.6: version GLIBC_2.32' not found (required by /usr/bin/promtail) /usr/bin/promtail: /lib/x86_64-linux-gnu/libc.so.6: version GLIBC_2.34' not found (required by /usr/bin/promtail)
I can confirm that I am seeing the same issue with the RPM repository when installing on AlmaLinux 8.10.
Can confirm the same on Ubuntu 20.04.6 LTS (Focal Fossa):
Jul 3 10:52:36 loki promtail[377]: /usr/bin/promtail: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /usr/bin/promtail)
A rollback/downgrade to promtail 3.0 is fixing it temporarily.
Please roll back any rpm+apt repositories distributing this binary. How can something like this just be distributed to a wide audience? Is there no testing for this stuff? This just breaks log aggregation for whole companies.
same issue on Rocky8
# promtail --version
promtail: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by promtail)
promtail: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by promtail)
Same issue here on Ubuntu 20
Edit: Working on Ubuntu 22 because GLIBC is version 2.35, but on Ubuntu 20, GLIBC is version 2.31
Good evening,
Issue is present with promtail-3.1.1-1.x86_64, so i went for an older version this package for the moment. In my case, for AlmaLinux release 8.10 (Cerulean Leopard) i have installed an older version: dnf install promtail-2.9.9-1.x86_64
CentOS 8 stream problem version 3.1.1 - /lib64/libc.so.6: version `GLIBC_2.32' not found
This affects Amazon Linux 2 as well (CentOS 7 derivative + updated kernel etc.)
Amazon Linux 2023 works ok.
EDIT: Downgrading to 2.9.9 works on older OSes.
Tried downgrading to 3.0.1 and 2.9.10 and finally got it working with 2.9.9 on Ubuntu 20
$ grep -i pretty /etc/os-release
PRETTY_NAME="Ubuntu 20.04.6 LTS"
$ dpkg -s promtail | grep Version
Version: 2.9.9
I assume the fix would be to use a build container/os of a lower glibc version!
Same here, install from binaries, downgrade from 2.9.10 to 2.9.9 as a workaround (Debian 11, x86_64).
Same Issue occurs to me.
My OS is Debian 11.7
My Kernel version is 5.10.0-23-amd64 #1 SMP Debian 5.10.179-1 (2023-05-12) x86_64 GNU/Linux
I check my GLIBC Version as belows.
ldd --version
My GLIBC Version is ldd (Debian GLIBC 2.31-13+deb11u6) 2.31
If you use a version of Promtail that is compatible with the GLIBC version of your OS, this may fix the problem.
@iljalukin In my case,
Promtail v2.9.9, It works well.
This does not solve my problem, as I want to install the latest available version via the APT repository as described above.
My workaround is to install version 3.0.0 with apt install promtail=3.0.0 and pin it with apt-mark hold promtail.
Debian 11 is the current LTS version, which will be maintained until 2026 and it would be nice if promtail could be installed there in the current version.
I hope that the developers will pay attention to this issue and fix it.
It's honestly a little concerning that this has not been fixed after two months. This breaks log aggregation on most enterprise Linux distributions used in production. I have decided to disabled the Grafana RPM repository completely and moved working versions of the RPMs that I need into a private repository to avoid these things affecting production. But if this is not fixed in a reasonable time, then Grafana Loki is not the right solution.
Indeed concerning, hopefully due to holiday, meanwhile running old version. If not fix in long term, will need to look for alternative
I think there is no different way except updating OS which support target GLIBC library.
In production, Because It is hard to update OS version, It is better to make binary file less dependent on libraries like GLIBC closely related to the operating system.
https://github.com/grafana/loki/issues/13393#issuecomment-2295819830
https://github.com/grafana/loki/issues/13393#issuecomment-2302676145
I am not a GitHub workflow expert, so I am not able to submit the relevant PR to downgrade the "build environment", but I believe that the workflow is building on the latest Ubuntu (based on a rudimentary search inside the .github/ directory) which would have a glibc >= 2.32
My use case is to support Amazon Linux 2, which uses glibc 2.26
$ cat /etc/*release
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"
SUPPORT_END="2025-06-30"
Amazon Linux release 2 (Karoo)
$ ldd --version
ldd (GNU libc) 2.26
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
$ rpm -qa | grep glibc-2
glibc-2.26-64.amzn2.0.2.x86_64
It would be helpful to have a web page that shows the available Promtail Binary versions for each OS version.
Same with Rocky 8.9. Surprised this has not been dealt with given its severity.
