logging-flume icon indicating copy to clipboard operation
logging-flume copied to clipboard

DEFAULT_AUTO_OFFSET_RESET should be "latest" according to official do…

Open hangc0276 opened this issue 9 years ago • 6 comments

In the official document, the default value of "kafka.consumer.auto.offset.reset" is "latest", but in the source code, "DEFAULT_AUTO_OFFSET_RESET" is "earliest".

hangc0276 avatar Nov 25 '16 07:11 hangc0276

I think the reset configuration for kafkachannel is kind of internal. Without it the channel would be able to guarantee message delivery. With "latest" it would skip a whole bunch of events when offset has to be reseted.

simonati avatar Nov 25 '16 16:11 simonati

I would recommend changing the docs instead of the source

simonati avatar Nov 25 '16 16:11 simonati

With "earliest" it would reset the offset when the consumer restarted or rebalanced, which will lead to message repeat consumption

hangc0276 avatar Nov 25 '16 16:11 hangc0276

Existing implementation achieves at least once message delivery. This doesn't guarantee that messages are delivered more than once (in fact it is a known downside). Exposing this configuration allows users to change the delivery characteristics of kafkachannel - if really needed - but making it default would break existing setups.

simonati avatar Nov 27 '16 11:11 simonati

Agree that on this we need to document the existing default value.

jholoman avatar Nov 28 '16 04:11 jholoman

Can one of the admins verify this patch?

asfgit avatar Aug 17 '18 13:08 asfgit