java-kafka-client icon indicating copy to clipboard operation
java-kafka-client copied to clipboard

Doesn't work with Kafka 3.0

Open cykl opened this issue 3 years ago • 1 comments

Using opentracing-contrib/java-kafka-client 0.15.0 with Kafka client >= 3.0.0 leads doesn't work:

2022-03-11T13:45:32.603Z [03608a62-cf2b-4e0d-ba76-85bec1224a9b-app-bfd8b27c-46e3-422f-a692-a6eff7ac2a7a-StreamThread-1] ERROR org.apache.kafka.streams.KafkaStreams - stream-client [03608a62-cf2b-4e0d-ba76-85bec1224a9b-app-bfd8b27c-46e3-422f-a692-a6eff7ac2a7a] Encountered the following exception during processing and Kafka Streams opted to SHUTDOWN_CLIENT. The streams client is going to shut down now. 
java.lang.AbstractMethodError: Receiver class io.opentracing.contrib.kafka.TracingKafkaConsumer does not define or inherit an implementation of the resolved method 'abstract java.util.OptionalLong currentLag(org.apache.kafka.common.TopicPartition)' of interface org.apache.kafka.clients.consumer.Consumer.
	at org.apache.kafka.streams.processor.internals.PartitionGroup.readyToProcess(PartitionGroup.java:143)
	at org.apache.kafka.streams.processor.internals.StreamTask.isProcessable(StreamTask.java:674)
	at org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:694)
	at org.apache.kafka.streams.processor.internals.TaskManager.process(TaskManager.java:1296)
	at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:784)
	at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:604)
	at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:576)

Root cause is that Kafka 3.0 added a bunch of new methods on Consumer interface and this project defines TracingKafkaConsumer which implement Consumer interface. When Kafka 3+ clients tries to use new methods, preceding error occurs.

Would it be possible to cut a new release compatible with Kafka 3+? I can write the PR, just want to check this project isn't dead first.

cykl avatar Mar 11 '22 14:03 cykl

Same issue - would be great if this could be implemented!

Matt007 avatar Nov 10 '22 16:11 Matt007