smallrye-reactive-messaging icon indicating copy to clipboard operation
smallrye-reactive-messaging copied to clipboard

Seek to beginning or reset offset to 0 to reconsume a Kafka topic

Open scspijker opened this issue 1 year ago • 2 comments

When we make a boo boo while consuming topics, sometimes we want to re-consume the whole topic. SmallRye Reactive Messaging does not support this (at least on Kafka?).

Our current approach is either: Implement a rebalance listener and maintain the offset and partitions ourselves (undesirable for such a small request) Change the consumer group and delete the old one (undesirable as we are on a managed cluster and need to ask another team to clean up consumer groups, also this makes for ugly consumer group ID’s).

Spring Kafka recently implemented seekToBeginning: https://docs.spring.io/spring-kafka/docs/2.6.10/api/org/springframework/kafka/listener/ConsumerSeekAware.ConsumerSeekCallback.html#seekToBeginning(java.lang.String,int)

We would love for SmallRye’s Reactive Messaging to support this as well, since this is the preferred consumer for Quarkus.

scspijker avatar Dec 05 '23 09:12 scspijker