Kafka exactly once consumer - vertical scaling

[ichrist-gr]

Hello folks,

we have a case where we need to scale (vertical) our exactly once consumer-producer chain. For example we have 10 partitions and we would like to read the kafka events from 1 pod in parallel.

But unfortunatelly after an investigation we found out the below piece of code that restricts us from scaling our kafka consumer.

List<KafkaConsumer<Object, Object>> consumers = clientService.getConsumers(channel);
if (consumers.isEmpty()) {
    throw KafkaExceptions.ex.unableToFindConsumerForChannel(channel);
} else if (consumers.size() > 1) {
    throw KafkaExceptions.ex.exactlyOnceProcessingNotSupported(channel);
}

Code exists here in line 85: KafkaTransactionsImpl

Are you planning to enable the vertical scaling feature for the kafka consumer for exactly-once feature?