Unable to get docker output

[mo36924]

Bug Report

Describe the bug I'm unable to get docker output when using the docker input. No log is output when using cgroup v2 and cgroupfs cgroup driver.

To Reproduce Run: sudo docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker/containers/:/var/lib/docker/containers/ -v /sys/fs/cgroup:/sys/fs/cgroup -ti fluent/fluent-bit:3.2 /fluent-bit/bin/fluent-bit -i docker -o stdout

[2024/12/27 07:39:10] [ info] [fluent bit] version=3.2.3, commit=44a2d0d43f, pid=1
[2024/12/27 07:39:10] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/12/27 07:39:10] [ info] [simd    ] disabled
[2024/12/27 07:39:10] [ info] [cmetrics] version=0.9.9
[2024/12/27 07:39:10] [ info] [ctraces ] version=0.5.7
[2024/12/27 07:39:10] [ info] [input:docker:docker.0] initializing
[2024/12/27 07:39:10] [ info] [input:docker:docker.0] storage_strategy='memory' (memory only)
[2024/12/27 07:39:10] [ info] [input:docker:docker.0] Detected cgroups v2
[2024/12/27 07:39:10] [ info] [sp] stream processor started
[2024/12/27 07:39:10] [ info] [output:stdout:stdout.0] worker #0 started

Your Environment

• Version used: Fluent Bit v3.2.3
• Configuration: -i docker -o stdout
• Docker version: Docker version 27.4.1

sudo docker info | grep 'Cgroup Version'
# Cgroup Version: 2
sudo docker info | grep 'Cgroup Driver'
# Cgroup Driver: cgroupfs

If you change to systemd's Cgroup Driver, logs will be output normally. /etc/docker/daemon.json

{
  "exec-opts": ["native.cgroupdriver=cgroupfs"]
}

↓

{
  "exec-opts": ["native.cgroupdriver=systemd"]
}

sudo systemctl restart docker.service

Run: sudo docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker/containers/:/var/lib/docker/containers/ -v /sys/fs/cgroup:/sys/fs/cgroup -ti fluent/fluent-bit:3.2 /fluent-bit/bin/fluent-bit -i docker -o stdout

[2024/12/27 07:41:52] [ info] [fluent bit] version=3.2.3, commit=44a2d0d43f, pid=1
[2024/12/27 07:41:52] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/12/27 07:41:52] [ info] [simd    ] disabled
[2024/12/27 07:41:52] [ info] [cmetrics] version=0.9.9
[2024/12/27 07:41:52] [ info] [ctraces ] version=0.5.7
[2024/12/27 07:41:52] [ info] [input:docker:docker.0] initializing
[2024/12/27 07:41:52] [ info] [input:docker:docker.0] storage_strategy='memory' (memory only)
[2024/12/27 07:41:52] [ info] [input:docker:docker.0] Detected cgroups v2
[2024/12/27 07:41:52] [ info] [sp] stream processor started
[2024/12/27 07:41:52] [ info] [output:stdout:stdout.0] worker #0 started
[0] docker.0: [[1735285312.468842561, {}], {"id"=>"062d9b937d1d", "name"=>"zealous_jennings", "cpu_used"=>76003, "mem_used"=>23629824, "mem_limit"=>18446744073709551615}]
[0] docker.0: [[1735285313.468766782, {}], {"id"=>"062d9b937d1d", "name"=>"zealous_jennings", "cpu_used"=>76820, "mem_used"=>23896064, "mem_limit"=>18446744073709551615}]
...

Looking at the code, it seems that the combination of cgroup v1 and systemd driver, and the combination of cgroup v2 and cgroupfs driver do not output logs.

ref: https://docs.docker.com/engine/containers/runmetrics/#find-the-cgroup-for-a-given-container

https://github.com/fluent/fluent-bit/blob/d77c06dc6cae0eefe534435eee0fb024e3dc1021/plugins/in_docker/docker.h#L46-L59

ref: https://github.com/fluent/fluent-bit/issues/3882