karafka-testing
karafka-testing copied to clipboard
Support producing messages at specific offset
trafficstars
Is your feature request related to a problem? Please describe.
I am porting ruby-kafka consumers to karafka, and some of our consumers make use of the consumed messages offset. Specifically, we store that as metadata on logs, errors, etc
The current implementation doesn't expose the offset at all. Messages use the current size of the messages produced so far, so the first one gets 0, 1, etc.
Describe the solution you'd like
karafka.produce(message_value, key: key) should support an offset option.
Describe alternatives you've considered
It might be enough to be able to set the initial offset offset, ie start from 1337 + number of messages, isntead of just the number of messages
Additional context
https://karafka-io.slack.com/archives/C027C6M2B0U/p1736529486973089