Invalid Credential Will Crash the Fluentd Worker

[yangjiel]

Is your feature request related to a problem? Please describe.

My environment is: td-agent 4.4.1 fluentd 1.15.2 (c32842297ed2c306f1b841a8f6e55bdd0f1cb27f) With the latest s3 plugin as for submitting this request.

Here is what I found just now, with the faulty credential, the s3 plugin will result in error and crash the worker0, worker0 will restart and retry then crash again. This has a negative impact on other plugins that's running on worker0. Sample logs:

2023-02-06 20:48:11 -0800 [error]: #0 unexpected error error_class=RuntimeError error="can't call S3 API. Please check your credentials or s3_region configuration. error = #<Aws::S3::Errors::InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.>"
  2023-02-06 20:48:11 -0800 [error]: #0 suppressed same stacktrace
2023-02-06 20:48:11 -0800 [error]: Worker 0 finished unexpectedly with status 1
2023-02-06 20:48:11 -0800 [info]: adding match pattern="**" type="copy"
2023-02-06 20:48:11 -0800 [warn]: #0 define <match fluent.**> to capture fluentd logs in top level is deprecated. Use <label @FLUENT_LOG> instead
2023-02-06 20:48:11 -0800 [info]: #0 starting fluentd worker pid=214943 ppid=214799 worker=0

Describe the solution you'd like

Is it possible to make it as a warning instead so that it doesn't block/crash the worker? Thanks.

Describe alternatives you've considered

Prompting error message is ok, as long as with faulty credentials, worker itself doesn't crash.

Additional context

I tested Amazon SQS, Amazon cloudwatch log plugins, with the faulty credentials, they will just prompt a warning message and the worker can keep running instead of crashing.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days