Support for external store for checkpoint.

[nageshs28]

Expected Behavior Azure Event hubs provide Kafka API compatibility and provides sdk for blob storage. https://learn.microsoft.com/en-us/java/api/overview/azure/messaging-eventhubs-checkpointstore-blob-readme?view=azure-java-stable Can spring-kafka has similar offering so that we can plugin server details and checkpoints are taken care of.

https://learn.microsoft.com/en-us/java/api/overview/azure/messaging-eventhubs-checkpointstore-blob-readme?view=azure-java-stable

Current Behavior Kafka creates its own store for offset mgmt.

Context

[garyrussell]

Spring for Apache Kafka is a pure Kafka library; I don't think it is appropriate to provide support for vendor extensions in the core project.

It is easy to use an external state store with Spring for Apache Kafka

• use AckMode.MANUAL and store the offset(s) in the external store instead of calling Acknowledgment.acknowledge()
• extend AbstractConsumerSeekAware and seek to the externally stored offset(s) when partitions are assigned.

Such an extension probably belongs in some other project; for example: https://spring.io/projects/spring-cloud-azure which is maintained by Microsoft.

If it makes sense to create such an extension and, if the extension requires any additional hooks in this project, we'd be more than happy to receive a generic pull request.

[artembilan]

The Event Hubs is a messaging broker. It is essentially not Apache Kafka. It is nice of Azure that they provide a compatibility with Kafka client, so there is not too much on the consumer/producer side to know that essentially we deal with Event Hubs. The blob store is really that messaging broker feature, nothing to do with the client lib. Since Spring for Apache Kafka is just a high-level API over plain Apache Kafka Client, I don't see how any external stores may belong to this project...

[nageshs28]

Thanks @garyrussell for quick revert. I wanted to know your opinions on feasibility with spring-kafka. Let me explore on suggestions provided and perform poc. Also went through below doc for reference.