Why not refresh metadata when got ErrNotLeaderForPartition

[ForeverSRC]

We got some ErrNotLeaderForPartition error after retrying for 3 times, I wonder why the producer doesn't refresh metadata when got that error. https://github.com/IBM/sarama/blob/d9abf3cfb14d41fbc059d0f36195603000b1aad2/async_producer.go#L1121-L1126

[dnwe]

@ForeverSRC thanks for raising, a good question!

Refreshing metadata in the producer retry case was brought in as part of the Idempotent producer changes, but it's not clear that this refresh needs to be gated for idempotent-only retries

[dnwe]

@ForeverSRC having updated my memory of this after re-reading the code, any retriabe error for a given broker producer should cause it to 'abandon' the broker, do a metadata refresh on the assumption it needs use a different broker producer because the topicpartition has moved:

https://github.com/IBM/sarama/blob/5f63a84f47c39bf08a1c276f1f6b5f1d754e9fc3/async_producer.go#L710-L724

You'd see this in sarama logger output as (e.g.,): 13:45:16.866724 producer/leader/my_topic/0 state change to [retrying-1] 13:45:16.866735 producer/leader/my_topic/0 abandoning broker 2

[ForeverSRC]

In my use case we got three ErrNotLeaderForPartition errors because the retry times is 3 and the interval is 100ms, so I'm not sure if the abandon logic works or not.

[github-actions[bot]]

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.