rust-rdkafka icon indicating copy to clipboard operation
rust-rdkafka copied to clipboard

Detailed logs for base producer poll error events

Open EddieWhi opened this issue 11 months ago • 1 comments

Hi Maintainers,

We have a number of mystery logs in our systems using rust-rdkafka along the lines of:

Ignored event 'Error' on base producer poll

I created this simple branch to get the error details out so we could diagnose and fix. The issue was fairly simple in the end (idle connections were being closed by AWS LBs, recommended settings for connections.max.idle.ms were missing). It would be great if I could get this (or similar) into a release someday to help us diagnose this kind of issue in the future.

I can see that this change could subtly impact current users as new error events will be pushed through ClientContext::error. If that's considered acceptable and manageable then great. Alternatively could have pushed the error detail through ClientContext::log which feels like it's less likely to cause issue although it's a little unsymmetrical?

Any thoughts and help getting this in would be greatly appreciated.

EddieWhi avatar Feb 13 '25 09:02 EddieWhi

Would this also be a good place to discuss and perhaps add support for error event callback in the ProducerContext trait? Knowing about producer side errors specially ones which are categorised as fatal would be helpful

rhishikeshj avatar Jun 05 '25 07:06 rhishikeshj

Thank you for your contribution. I believe this was done as part of https://github.com/fede1024/rust-rdkafka/pull/771

fede1024 avatar Jun 22 '25 08:06 fede1024