vivekchavan14/fix autobalancer metrics reporter outofordersequence

[vivekchavan14]

Problem

Fixes #2697 - AutoBalancer metrics reporter experiencing OutOfOrderSequenceException and missing consumption data curves after upgrading from v1.1.2 to v1.4.1.

Root Cause

The producer was configured with retries and acks=all but missing enable.idempotence=true, causing sequence number conflicts during retries.

Changes Made

• Enable producer idempotence to prevent OutOfOrderSequenceException
• Add proper timeout configurations (delivery_timeout_ms=120000)
• Improve error handling with specific exception type logging
• Implement graceful shutdown with producer flush
• Add shutdown guards to prevent metrics sending during shutdown
• Enhanced tests for configuration validation

[superhx]

Welcome to use AI to fix issues, but the expected fixes need to be verified and confirmed by the developers themselves. @vivekchavan14

[vivekchavan14]

Yup, I’ve reviewed the response, it's not entirely AI-generated. I used AI to assist with the analysis, but all suggested fixes and insights have been manually verified and cross-checked before posting. Let me know if any part needs further clarification or adjustment.

[github-actions[bot]]

This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge conflicts, please update it with the latest from trunk (or appropriate release branch)

If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed.