Dominic Evans
Dominic Evans
@tomplarge thanks for raising this one. When we commit the transaction, the txnmgr/endtxn request is sent to the coordinator, which writes a `PrepareCommit` message to the transaction log and then...
Ah, I just spotted your `Retry.Max = 1` and `Retry.Backoff = 1 * time.Millisecond`, so that's probably why the existing Retry code isn't working for you here https://github.com/IBM/sarama/blob/4ad35041300e1c15ba58b630745ac8eb05f30c10/transaction_manager.go#L359-L360
@tomplarge yeah I agree it shouldn't be async, ideally we wouldn't get the response from the Commit until it had 100% completed server side, but that async behaviour seems to...
@shubham-dogra-s1 👋🏻 thanks for getting in touch The first thing to double check would be your consumer group lag vs the topic retention. If the group is too far behind...
@road-cycling thanks for reporting this, would it be possible for you to upgrade to a newer version of Sarama? At least v1.41.1, but ideally picking up the latest v1.42.2. In...
@R4wm there's nothing obviously wrong in the goroutine dump there, did you have an associated panic? Sarama v1.27.0 is pretty old now (2020-08-11) so it would be worth you upgrading...
So this appears to be an ordering issue / race condition between new batches being produced and batches being retried in the idempotent producer: https://github.com/IBM/sarama/blob/f21c5125746f9d10fd731dfdff54a494098626d1/async_producer.go#L1144-L1148 This shouldn't occur with `config.Net.MaxOpenRequests...
@ForeverSRC thanks for raising, a good question! Refreshing metadata in the producer retry case was brought in as part of the Idempotent producer changes, but it's not clear that this...
@ForeverSRC having updated my memory of this after re-reading the code, any retriabe error for a given broker producer should cause it to 'abandon' the broker, do a metadata refresh...
@omersiar Sarama does send its software name and version to Kafka as long as you are using sarama.Version = V2_4_0_0 or newer in your config and haven't disabled it via...