fix: Added broker connection check in metadata refresh loop. Closes broken broker connections

[doxsch]

I have implemented a workaround here. The solution is not optimal but makes the admin client usable for long term executions.

During the metadata request all brokers, which have an open connection, are checked by an ApiVersionRequest if it still works. If this is not the case, the connection is closed and reopened with the next request.

This way with the refresh interval < 10min or better said < min(connections.max.idle.ms, keepAlive kernel settings) we can achieve that the connections are no longer closed by inactivity and if this should happen anyway, the cached connection will be closed.

In v2 we should take into account that a TCP connection can also be closed remotely.

closes #1796

[look]

We are also experiencing this error, so 👍 to fixing it. As a work around, I'm investigating re-creating the ClusterAdmin every time we use it to avoid idle time outs.

[VipulZopSmart]

Still experiencing this error in the application, is it got fixed? @doxsch @look

[VipulZopSmart]

Please kindly help me with some solution, as it's taking much time to form that broker connection, it's affecting response time of my application. @look @doxsch. Please do let me know if there's any any config that i can change in the sarama to atleast increase this.

[mpavlikWandera]

@bai @dnwe could you pls look at the review?

[mpavlikWandera]

Bump ... pretty please have a look at this @bai @dnwe

[github-actions[bot]]

Thank you for your contribution! However, this pull request has not had any activity in the past 90 days and will be closed in 30 days if no updates occur. If you believe the changes are still valid then please verify your branch has no conflicts with main and rebase if needed. If you are awaiting a (re-)review then please let us know.

[github-actions[bot]]

Thank you for your contribution! However, this pull request has not had any activity in the past 90 days and will be closed in 30 days if no updates occur. If you believe the changes are still valid then please verify your branch has no conflicts with main and rebase if needed. If you are awaiting a (re-)review then please let us know.