rust-rdkafka icon indicating copy to clipboard operation
rust-rdkafka copied to clipboard

Improve documentation for Commit Offset semantics

Open aarashy opened this issue 2 years ago • 1 comments

Hello,

Let me start with a quick thank you to all who maintain this Rust crate!

I had been struggling with an off-by-one error which caused my consumer service to double-consume the last message any time the server restarted, when I realized that I misunderstood the semantics of Commit Offsets.

I found this thread which finally clarified for me that I need to commit the offset of the next message that I want to receive, NOT the offset of the most recently consumed message.

I am very surprised at how difficult it was to find a clear description of this behavior in all of the online Kafka documentation, and I thought I may save somebody else a headache if I call this out explicitly in your documentation.

Feel free to make any additional edits. Thanks again!

aarashy avatar May 17 '22 19:05 aarashy

Hi @duarten @benesch - do you have any comment on this documentation improvement?

aarashy avatar Jul 11 '22 17:07 aarashy

I am very surprised at how difficult it was to find a clear description of this behavior in all of the online Kafka documentation, and I thought I may save somebody else a headache if I call this out explicitly in your documentation.

Ha! My coworker at @MaterializeInc (@guswynn) and I both recently got frustrated by how hard it is to find an official reference for this behavior too! Very happy to have this in the docs here. Thanks for writing up the PR!

benesch avatar Oct 29 '22 16:10 benesch