Permission denied with fluent-cap-ctl and dac_read_search

[rayate2410]

Describe the bug

I have a k8s deployment of fluentd, which is supposed to tail the logs for all containers. For security reasons, we want to run the fluentD with non-root user (fluent) .

Container log permissions don't allow to be able to read by any other than root user. drwx------ 1 root root 7680 Jun 22 13:33 containers

I was trying to see if I can use linux capabilities mentioned here.

With above image, when I try to run fluentd, I'm getting permission denied error.

With dac_read_search, I'm getting permission denied error. With dac_override, it is working as expected

To Reproduce

Below is my custom docker file.

FROM fluent/fluentd:v1.16.0-1.0

# Use root account to use apk
USER root

RUN apk update && apk upgrade && apk add --no-cache --update --virtual .build-deps \
        build-base ruby-dev make pkgconfig \
 && gem install fluent-plugin-multi-format-parser \
 && gem install fluent-plugin-kubernetes_metadata_filter \
 && gem install fluent-plugin-detect-exceptions \
 && gem install fluent-plugin-elasticsearch \
 && gem install fluent-plugin-grafana-loki \
 && gem install fluent-plugin-record-modifier \
 && apk add --no-cache --update libcap-ng libcap-ng-dev \
 && gem install capng_c \
 && gem sources --clear-all \
 && apk del .build-deps \
 && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem

RUN fluent-cap-ctl --add dac_read_search -f /usr/bin/ruby
# RUN fluent-cap-ctl --add dac_override -f /usr/bin/ruby

USER fluent

Once image is built, I'm trying to run simple fluentd command.

$ docker run -it --rm em-fluentd:localtest sh
/ $ id
uid=100(fluent) gid=101(fluent) groups=101(fluent)
/ $ fluentd --version
sh: fluentd: Operation not permitted
/ $

Expected behavior

fluentd should work after setting the capabilities.

Your Environment

- Fluentd version: v1.16.0-1.0
- TD Agent version:
- Operating system: Alpine Linux v3.17 
- Kernel version: 5.14.21-150400.24.18-default

Your Configuration

<source>
      @type tail
      path /var/log/containers/*.log
      exclude_path ["/var/log/containers/eml-fluentd**"]
      pos_file /var/log/fluentd/tmp/containers-access.log.pos
      read_from_head true
      # we add this tag because the multiline plugin forces us to remove some tag after processing
      tag removeme.kubernetes.*
      <parse>
        @type multi_format
        <pattern>
          format json
          time_key time
          time_type string
          time_format "%Y-%m-%dT%H:%M:%S.%NZ"
          keep_time_key true
        </pattern>
        <pattern>
          format regexp
          expression /^(?<time>.+) (?<stream>stdout|stderr)( (?<logtag>.))? (?<log>.*)$/
          time_key time
          time_format '%Y-%m-%dT%H:%M:%S.%N%:z'
          keep_time_key true
        </pattern>
      </parse>
    </source>

Your Error Log

$ docker run -it --rm em-fluentd:localtest sh
/ $ id
uid=100(fluent) gid=101(fluent) groups=101(fluent)
/ $ fluentd --version
sh: fluentd: Operation not permitted
/ $

Additional context

No response