fluent
RESOLVED: Workaround: disable jemalloc when image crashed (segmentation fault) with systemd plugin
Describe the bug
There is a known issue that combination with systemd plugin and jemalloc causes a crash bug.
There are some feedback from users.
Related issue:
- https://github.com/fluent/fluent-package-builder/issues/369
- https://github.com/fluent-plugins-nursery/fluent-plugin-systemd/issues/110
- https://github.com/fluent/fluentd-docker-image/issues/378
[!IMPORTANT] As a workaround, you can avoid a segmentation fault to disable jemalloc memory allocator by specifying empty LD_PRELOAD for each container.
docker run --rm -it -e LD_PRELOAD="" ....
[!CAUTION] UPDATE: 2024 Sep 11 fluent-plugin-systemd 1.1.0 (systemd-journal 2.0.0) was shipped to fix this issue. Please use fluent-plugin-systemd 1.1.0 or later.
To Reproduce
See above mentioned issues and related bugs. (Especially https://github.com/fluent/fluentd-docker-image/issues/378 is the simplified to reproduce without k8s)
Expected behavior
Even though jemalloc is enabled, it does not cause a segmentation fault)
Your Environment
- all of v1.16.x or later
Your Configuration
See already mentioned issues.
Your Error Log
See already mentioned issues.
Additional context
To avoid this issue, jemalloc memory allocator will be disabled by default for v1.17.0-1.3 or later and v1.16.5-1.3 or later for fluent-kubernetes-daemonset images.
If you don't use systemd plugin at all, no need to disable jemalloc memory allocator explicitly.
NOTE: recent version of jemalloc is not solution for us. checked with:
- jemalloc c893fcd169fffca1b9d3156c6637a197765b82d0
- fluentd v1.17.0
- systemd-journal 1.4.2
- fluent-plugin-systemd 1.0.5
This issue was fixed via fluent-plugin-systemd 1.1.0 (which uses systemd-journal 2.0.0)
Now no need to disable jemalloc explicitly.
Please use fluent-plugin-systemd 1.1.0 or later.
ref. https://github.com/ledbettj/systemd-journal/pull/96 https://github.com/ledbettj/systemd-journal/pull/97 https://github.com/fluent/fluentd-docker-image/issues/378
[!NOTE] Keep this issue open for a while to inform you.
3 months had passed since the issue was created to raise an attention.
It's time to close it.