fluent-plugin-aws-elasticsearch-service icon indicating copy to clipboard operation
fluent-plugin-aws-elasticsearch-service copied to clipboard

Getting "undefined method `[]`" when using this with ES 6.0 on AWS

Open daviddyball opened this issue 6 years ago • 3 comments

Trying to post to an ES 6.0 domain in AWS and getting the following error. I can't find any information on the undefined method \[]`` error it's complaining about though. Is there any additional logging I can enable to make debugging this any easier?

2018-03-01 19:20:39 +0000 [info]: #0 fluentd worker is now running worker=0
2018-03-01 19:20:44 +0000 [warn]: #0 failed to flush the buffer. retry_time=0 next_retry_seconds=2018-03-01 19:20:45 +0000 chunk="5665ec1468f652a7261c16755f23a988" error_class=NoMethodError error="undefined method `[]' for nil:NilClass"
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/excon-0.60.0/lib/excon/connection.rb:244:in `request'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/faraday-0.14.0/lib/faraday/adapter/excon.rb:49:in `call'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/faraday_middleware-aws-signers-v4-0.1.1/lib/faraday_middleware/request/aws_signers_v4.rb:41:in `call'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/faraday-0.14.0/lib/faraday/rack_builder.rb:143:in `build_response'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/faraday-0.14.0/lib/faraday/connection.rb:387:in `run_request'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/elasticsearch-transport-6.0.1/lib/elasticsearch/transport/transport/http/faraday.rb:23:in `block in perform_request'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/elasticsearch-transport-6.0.1/lib/elasticsearch/transport/transport/base.rb:262:in `perform_request'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/elasticsearch-transport-6.0.1/lib/elasticsearch/transport/transport/http/faraday.rb:20:in `perform_request'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/elasticsearch-transport-6.0.1/lib/elasticsearch/transport/client.rb:131:in `perform_request'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/elasticsearch-api-6.0.1/lib/elasticsearch/api/actions/ping.rb:20:in `ping'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-elasticsearch-2.0.1/lib/fluent/plugin/out_elasticsearch.rb:191:in `client'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-elasticsearch-2.0.1/lib/fluent/plugin/out_elasticsearch.rb:425:in `rescue in send_bulk'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-elasticsearch-2.0.1/lib/fluent/plugin/out_elasticsearch.rb:420:in `send_bulk'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-elasticsearch-2.0.1/lib/fluent/plugin/out_elasticsearch.rb:407:in `write'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-aws-elasticsearch-service-1.0.0/lib/fluent/plugin/out_aws-elasticsearch-service.rb:61:in `write'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.14.25/lib/fluent/plugin/output.rb:1093:in `try_flush'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.14.25/lib/fluent/plugin/output.rb:1318:in `flush_thread_run'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.14.25/lib/fluent/plugin/output.rb:439:in `block (2 levels) in start'
  2018-03-01 19:20:44 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.14.25/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'

daviddyball avatar Mar 01 '18 19:03 daviddyball

Linking to https://github.com/atomita/fluent-plugin-aws-elasticsearch-service/issues/41 which is a general query about 6.0 support in the plugin.

daviddyball avatar Mar 01 '18 19:03 daviddyball

I will try spinning up an ES 5.5 cluster and see if my configuration gets the same errors.

daviddyball avatar Mar 01 '18 19:03 daviddyball

Nope, same with a 5.5 cluster. Any help is appreciated.

daviddyball avatar Mar 01 '18 20:03 daviddyball