[Question] Consumer, Non-destructive flush of final stored offsets when EnableAutoOffsetStore = false and EnableAutoCommit = true

[ycherkes]

I'm trying to use the recommended asynchronous commit approach documentation, but I ran into the problem that the only way to forcibly commit all the final stored offsets is with Consumer.Close().

But this is a destructive call, is there a way to do this without destroying the consumer?

I've tried to call the consumer.Commit(), sometimes it raises an exception (no stored offsets), sometimes it runs with no errors, but it doesn't cause consumer.OffsetsCommitted event.

[mhowlett]

the no stored offsets exception will be harmless, but it shouldn't be exposed. Commit should work. what version are you using?

[ycherkes]

Hi @mhowlett, I'm using the following version: Confluent.Kafka 1.8.2. As far as I understand, this question is closer to librdkafka.redist project, because they don't provide the FlushStoredOffsets method. I wouldn't use the Commit method, because this method doesn't fire OffsetsCommitted events, it breaks the consistency of the handling of OffsetsCommitted events. The Close method fires this kind of event in opposite to the Commit method. But it has a side effect (consumer destroying).

[mhowlett]

What is your use case for FlushStoredOffsets? The API can't do what you want, and it would be probably better if it could - I'm not sure what the priority is though, given I don't know when you would use this.

[mhowlett]

i'm closing this as below the threshold for something we'll likely act on, but feel free to continue the conversation.