fluent-plugin-kafka icon indicating copy to clipboard operation
fluent-plugin-kafka copied to clipboard

out_rdkafka2: Handle some of the exceptions as unrecoverable errors

Open cosmo0920 opened this issue 1 year ago • 1 comments
trafficstars

This is because msg_size_too_large and topic_authorization_failed error codes should be treated as an unrecoverable error. In Fluentd, we need to mark as unrecoverable with raise Fluent::Unrecoverable. But current implementation does not handle them as unrecoverable errors.

cosmo0920 avatar Jun 25 '24 08:06 cosmo0920

FYI: we use flog to generate arbitrary length of logs. In this case, we use flog -b 1024000 -n 1 to create too huge line to consume by out_rdkafka2 plugin.

cosmo0920 avatar Jun 26 '24 10:06 cosmo0920

Thanks!

ashie avatar Jul 10 '24 23:07 ashie