Masahiro Nakagawa

@daic-h If you have more better solution to fix this and #16, patches are welcome!

Their requirement seems send events to multiple topics in one flush, not create outputs for each topic. Hmm... we need to check ruby-kafka supports it.

Does anyone have an idea for this error? On my environment, `` works corretly and I first see "unable to sign request without credentials set" error. So I'm not sure...

We need help from AWS expert to investigate deeply. For example, this error happens when no `~/.aws/credentials` and no aws related environment variable. I'm not sure full spec of AWS...

For that case, the error should be AccessDenied. Did you get MissingCredentialsError instead of AccessDenied with lack of permission?

Your metric is very strange. MongoDB operations metric shows no operation but your network usage is not decreased. No warning or error logs at 8:40 - 8:50? If the bottleneck...

> 2020-06-16 13:24:11 +0000 [warn]: #0 [out_s3] buffer flush took longer time than slow_flush_log_threshold: elapsed_time=35.5713529381901 slow_flush_log_threshold=20.0 plugin_id="out_s3" This means uploading data to S3 took 35 seconds. I'm not sure why......

fluentd's in_forward can accept multiple events at once so this method could be implemented: https://github.com/fluent/fluentd/blob/master/lib/fluent/plugin/in_forward.rb#L104-L113

JSON support should be separated to other PR.

It depends on Python's socket implementation. If CPython, PyPy and other implmentaions are safe, we can write "This is fork-safe."