fs2-kafka icon indicating copy to clipboard operation
fs2-kafka copied to clipboard

kafka 0.11.0 is not supported

Open er999mi opened this issue 7 years ago • 3 comments

er999mi avatar Aug 06 '17 14:08 er999mi

We will look into that. However releases of kafka are backward compatible, so you may use the driver and talk to 0.11 ensemble like it would by 0.10.x ensemble.

pchlupacek avatar Aug 07 '17 04:08 pchlupacek

One really cool feature of the newest Kafka versions is ability to have the record headers (https://cwiki.apache.org/confluence/display/KAFKA/KIP-82+-+Add+Record+Headers).

This, for example, allows one to filter the messages quickly without the need to parse the body of the message.

cleverdeveloper avatar Jan 28 '19 14:01 cleverdeveloper

This is quite interesting feature. We are doing similar thing with lazy evaluation and memoization of that evaluation. In fact if you think about it, this is available if you use cats.Eval, and define own structure of messages.

From the document I assume, that you have to always read full message bytes, so adding headers will slightly hurt performance /though likely non-measurable way).

As for the implementation, I believe only scodec kafka protocols have to be changed to support this feature.

pchlupacek avatar Jan 28 '19 14:01 pchlupacek