fluent-plugin-aws-elasticsearch-service icon indicating copy to clipboard operation
fluent-plugin-aws-elasticsearch-service copied to clipboard

Error starting FluentD

Open AnakinPt opened this issue 2 years ago • 2 comments

Problem

When I start the fluentD with this plugin, I get an error
2021-08-06 16:36:11 +0000 [info]: gem 'fluentd' version '1.13.3'
2021-08-06T16:36:12.215656300Z /usr/local/bundle/gems/fluent-plugin-aws-elasticsearch-service-2.4.1/lib/fluent/plugin/out_aws-elasticsearch-service.rb:144:in `include': wrong argument type Class (expected Module) (TypeError)
2021-08-06T16:36:12.215693000Z 	from /usr/local/bundle/gems/fluent-plugin-aws-elasticsearch-service-2.4.1/lib/fluent/plugin/out_aws-elasticsearch-service.rb:144:in `<module:Client>'
2021-08-06T16:36:12.215695800Z 	from /usr/local/bundle/gems/fluent-plugin-aws-elasticsearch-service-2.4.1/lib/fluent/plugin/out_aws-elasticsearch-service.rb:143:in `<module:Elasticsearch>'
2021-08-06T16:36:12.215698000Z 	from /usr/local/bundle/gems/fluent-plugin-aws-elasticsearch-service-2.4.1/lib/fluent/plugin/out_aws-elasticsearch-service.rb:141:in `<class:ElasticsearchOutput>'
2021-08-06T16:36:12.215710700Z 	from /usr/local/bundle/gems/fluent-plugin-aws-elasticsearch-service-2.4.1/lib/fluent/plugin/out_aws-elasticsearch-service.rb:140:in `<module:Plugin>'
2021-08-06T16:36:12.215712900Z 	from /usr/local/bundle/gems/fluent-plugin-aws-elasticsearch-service-2.4.1/lib/fluent/plugin/out_aws-elasticsearch-service.rb:9:in `<top (required)>'
2021-08-06T16:36:12.215714900Z 	from /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
2021-08-06T16:36:12.215716600Z 	from /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
2021-08-06T16:36:12.215718300Z 	from /usr/local/bundle/gems/fluentd-1.13.3/lib/fluent/registry.rb:102:in `block in search'
2021-08-06T16:36:12.215720000Z 	from /usr/local/bundle/gems/fluentd-1.13.3/lib/fluent/registry.rb:99:in `each'
2021-08-06T16:36:12.215721700Z 	from /usr/local/bundle/gems/fluentd-1.13.3/lib/fluent/registry.rb:99:in `search'
2021-08-06T16:36:12.215723400Z 	from /usr/local/bundle/gems/fluentd-1.13.3/lib/fluent/registry.rb:44:in `lookup'
2021-08-06T16:36:12.215725100Z 	from /usr/local/bundle/gems/fluentd-1.13.3/lib/fluent/plugin.rb:160:in `new_impl'
2021-08-06T16:36:12.215726700Z 	from /usr/local/bundle/gems/fluentd-1.13.3/lib/fluent/plugin.rb:109:in `new_output'
2021-08-06T16:36:12.215728500Z 	from /usr/local/bundle/gems/fluentd-1.13.3/lib/fluent/agent.rb:130:in `add_match'
2021-08-06T16:36:12.215730200Z 	from /usr/local/bundle/gems/fluentd-1.13.3/lib/fluent/agent.rb:74:in `block in configure'
2021-08-06T16:36:12.215731900Z 	from /usr/local/bundle/gems/fluentd-1.13.3/lib/fluent/agent.rb:64:in `each'
2021-08-06T16:36:12.215733600Z 	from /usr/local/bundle/gems/fluentd-1.13.3/lib/fluent/agent.rb:64:in `configure'
2021-08-06T16:36:12.215736000Z 	from /usr/local/bundle/gems/fluentd-1.13.3/lib/fluent/root_agent.rb:146:in `configure'
2021-08-06T16:36:12.215737800Z 	from /usr/local/bundle/gems/fluentd-1.13.3/lib/fluent/engine.rb:105:in `configure'
2021-08-06T16:36:12.215739400Z 	from /usr/local/bundle/gems/fluentd-1.13.3/lib/fluent/engine.rb:80:in `run_configure'
2021-08-06T16:36:12.215741100Z 	from /usr/local/bundle/gems/fluentd-1.13.3/lib/fluent/supervisor.rb:663:in `run_supervisor'
2021-08-06T16:36:12.215742900Z 	from /usr/local/bundle/gems/fluentd-1.13.3/lib/fluent/command/fluentd.rb:344:in `<top (required)>'
2021-08-06T16:36:12.215744800Z 	from /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
2021-08-06T16:36:12.215746800Z 	from /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
2021-08-06T16:36:12.215748600Z 	from /usr/local/bundle/gems/fluentd-1.13.3/bin/fluentd:15:in `<top (required)>'
2021-08-06T16:36:12.215750300Z 	from /usr/local/bundle/bin/fluentd:23:in `load'
2021-08-06T16:36:12.215751900Z 	from /usr/local/bundle/bin/fluentd:23:in `<main>'

Steps to replicate

I'm using this Dockerfile: FROM fluent/fluentd:edge-debian

USER root RUN ["gem", "install", "fluent-plugin-aws-elasticsearch-service"] RUN ["gem", "install", "fluent-plugin-record-modifier"] RUN ["gem", "install", "fluent-plugin-http-healthcheck"] RUN ["apt-get", "update"] RUN ["apt-get", "-y", "install", "curl"]

HEALTHCHECK --interval=30s --timeout=3s CMD curl -v http://localhost:8888 || exit 1

COPY entrypoint.sh /bin/

USER fluent

Provide example config and message

<match gelf-logging>
  @type "aws-elasticsearch-service"
  type_name "access_log"
  logstash_format true
  include_tag_key true
  tag_key "@log_name"
  logstash_prefix dummy
  templates { "dummy": "/fluentd/etc/dummy.json" }
  flush_interval 10s

  <endpoint>
    url "https://myElasticSearchInAws.eu-central-1.es.amazonaws.com"
    region "eu-central-1"
  </endpoint>
</match>

Expected Behavior or What you need to ask

I expect the application to start normally ...

Using Fluentd and ES plugin versions

Everything was provided in the error

AnakinPt avatar Aug 06 '21 16:08 AnakinPt

Same issue here. Started to happen friday. There is no new version of the ES plugin so i don't why it started to happen out of the blue like that.

bloodguard avatar Aug 09 '21 15:08 bloodguard

@AnakinPt Found the issue. I would recommend to read this issue: https://github.com/uken/fluent-plugin-elasticsearch/issues/912

bloodguard avatar Aug 09 '21 16:08 bloodguard