kaf icon indicating copy to clipboard operation
kaf copied to clipboard

Is there any format that is guaranteed to work for kaf consume | kaf produce pipelines?

Open cobbr2 opened this issue 1 year ago • 0 comments

I see the issue for implementing kaf cp, but my question is simpler, I think: is there any specific combination of options for the existing kaf which we know will "just work" when doing something like:

kaf consume -c cluster1 my_topic | kaf produce -c cluster2 my_topic_in_cluster2

?

I've tried a bunch of combinations, but I haven't hit a workable one. Our messages are protobufs, and have a headers and key entry in addition to the payload element when output in JSON; most of my solutions so far drop the headers and key entirely. The ones that don't wrap the whole message into a new "payload". I want to preserve all three.

cobbr2 avatar Dec 04 '24 22:12 cobbr2