Error Starting Fluentd

[wooxudong]

Problem

The fluend container would not be able to start with following errors ...

Steps to replicate

FROM bitnami/fluentd:1.14.0

RUN fluent-gem install 'fluent-plugin-aws-elasticsearch-service' -v 2.4.1

Expected Behavior or What you need to ask

Fluentd to start correctly ...

Using Fluentd and ES plugin versions

Fluentd: v1.14.0 fluent-plugin-aws-elasticsearch-service: v 2.4.1

[pavel-trubitsyn]

Hi, I have the same issue with fluent/fluentd:v1.14.1.

[pavel-trubitsyn]

Solution https://github.com/uken/fluent-plugin-elasticsearch/issues/912#issuecomment-894242260

[wooxudong]

@pavel-trubitsyn , thanks for the solution.

I think it shall be a proper fix for this plugin to solve the dependency chain ?

[wooxudong]

Any updates on this ?

[DaemonDude23]

I've been watching this problem overall for the last year. Today I got it working after learning about an OpenSearch-specific plugin from the Fluentd maintainers.

I had previously built a container with plugins from this repo to push logs to AWS's OpenSearch, but when trying to update it, ran into these dependency conflicts. I upgraded from: fluent/fluentd:v1.13.3-debian-1.0 to fluent/fluentd:v1.14.5-debian-1.0. The Dockerfile I kept the same, other than removing any elasticsearch plugins and replacing it with: fluent-plugin-opensearch:1.0.1 (https://github.com/fluent/fluent-plugin-opensearch).

Then I modified the config to change the output @type aws-elasticsearch-service to opensearch. I removed 2 parameters that are not supported by this plugin (log_es_400_reason and suppress_type_name) and it's working just fine. I see there are many issues in this repo that relate to this problem. Hopefully this will help someone resolve it. EDIT: There's a bug with the Opensearch plugin: https://github.com/fluent/fluent-plugin-opensearch/issues/24