fluentd
fluentd copied to clipboard
-p overwrites the default plugin path
Describe the bug
When updating from fluent/fluentd-kubernetes-daemonset:v1.14.6-debian-logzio-amd64-1.0
to fluent/fluentd-kubernetes-daemonset:v1.16.2-debian-logzio-amd64-1.0
our setup stopped working.
During investigation it turned out that plugins in the default plugin directory (/etc/fluent/plugin/
) are not processed anymore when using the -p
parameter.
To Reproduce
Add custom pluins into /etc/fluent/plugin/
directory and ensure that they are used by the config:
root@27cf7d7bb6eb:/home/fluent# ls -1 /etc/fluent/plugin/
filter_empty_field.rb
filter_field_size.rb
filter_field_type.rb
filter_message.rb
When starting fluentd with the default options from the Dockerfile command (fluentd -c /test/fluentd.conf -p /fluentd/plugins/ --gemfile /fluentd/Gemfile
) I am getting this error:
2023-08-16 10:20:01 +0000 [error]: config error file="/test/fluentd.conf" error_class=Fluent::NotFoundPluginError error="Unknown filter plugin 'message'. Run 'gem search -rd fluent-plugin' to find plugins"
I can get rid of this error by either not specifying -p
at all:
fluentd -c /test/fluentd.conf --gemfile /fluentd/Gemfile
or by adding the default plugin explicitly:
fluentd -c /test/fluentd.conf -p /fluentd/plugins/ -p /etc/fluent/plugin --gemfile /fluentd/Gemfile
Expected behavior
The -p
parameter should add plugin directories as documented without unsetting the default directory.
Your Environment
- Fluentd version: `1.16.2`
- Operating system: `Debian 11`
- Kernel version: `5.15.49-linuxkit-pr`
Your Configuration
<label @FLUENT_LOG>
<match fluent.**>
@type null
</match>
</label>
<source>
@type tail
@id in_tail_container_logs
path /var/log/containers/*.log
tag "kubernetes.*"
pos_file /var/log/containers/fluentd-containers.log.pos
read_from_head true
<parse>
@type "json"
time_format %Y-%m-%dT%H:%M:%S.%NZ
</parse>
</source>
@include /fluentd/etc/conf.d/*.conf
[..]
/fluentd/etc/conf.d/01-pre-parsing-filters.conf:
<filter kubernetes.**>
@type message
message_fields log, MESSAGE
</filter>
### Your Error Log
```shell
2023-08-16 10:27:01 +0000 [info]: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2023-08-16 10:27:01 +0000 [warn]: '@' is the system reserved prefix. It works in the nested configuration for now but it will be rejected: @timestamp
2023-08-16 10:27:01 +0000 [info]: parsing config file is succeeded path="/test/fluentd.conf"
2023-08-16 10:27:02 +0000 [info]: gem 'fluentd' version '1.16.2'
2023-08-16 10:27:02 +0000 [info]: gem 'fluent-plugin-concat' version '2.5.0'
2023-08-16 10:27:02 +0000 [info]: gem 'fluent-plugin-detect-exceptions' version '0.0.15'
2023-08-16 10:27:02 +0000 [info]: gem 'fluent-plugin-grok-parser' version '2.6.2'
2023-08-16 10:27:02 +0000 [info]: gem 'fluent-plugin-json-in-json-2' version '1.0.2'
2023-08-16 10:27:02 +0000 [info]: gem 'fluent-plugin-kubernetes_metadata_filter' version '3.2.0'
2023-08-16 10:27:02 +0000 [info]: gem 'fluent-plugin-logzio' version '0.2.1'
2023-08-16 10:27:02 +0000 [info]: gem 'fluent-plugin-multi-format-parser' version '1.0.0'
2023-08-16 10:27:02 +0000 [info]: gem 'fluent-plugin-parser-cri' version '0.1.1'
2023-08-16 10:27:02 +0000 [info]: gem 'fluent-plugin-prometheus' version '2.1.0'
2023-08-16 10:27:02 +0000 [info]: gem 'fluent-plugin-record-modifier' version '2.1.1'
2023-08-16 10:27:02 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.4.0'
2023-08-16 10:27:02 +0000 [info]: gem 'fluent-plugin-systemd' version '1.0.5'
2023-08-16 10:27:02 +0000 [error]: config error file="/test/fluentd.conf" error_class=Fluent::NotFoundPluginError error="Unknown filter plugin 'message'. Run 'gem search -rd fluent-plugin' to find plugins"
### Additional context
_No response_