kafka-connect-zeebe icon indicating copy to clipboard operation
kafka-connect-zeebe copied to clipboard

Use Kafka Connect "Single Message Transforms" instead of custom JsonPath

Open berndruecker opened this issue 5 years ago • 1 comments

We currently use some JsonPath ourselves to configure where to read certain data from, see e.g. https://github.com/zeebe-io/kafka-connect-zeebe/blob/master/examples/ping-pong/sink.json#L13

That also means we only support Json, not Avro

It would be more Kafka Connect style, if we use the “transforms” (https://docs.confluent.io/current/connect/transforms/index.html) – as in the example in https://www.confluent.io/blog/kafka-connect-deep-dive-jdbc-source-connector

Then we could simply define the expected message format and provide some sample transforms to get there.

berndruecker avatar Sep 02 '19 10:09 berndruecker

Hi, As of current version of the zeebe kafka connect plugin, is it possible to use built-in SMT like https://docs.confluent.io/platform/current/connect/transforms/extractfield.html to change the kafka messages being sent from source connector?

ankit-joinwal avatar Aug 06 '21 10:08 ankit-joinwal