kafka-streams
kafka-streams copied to clipboard
Merge does not produce to output stream. but to respective. input stream
I am having a really weird issue where the input streams are being written instead of target topic. I am running the code example:
const stream1 = kafkaStreams.getKStream("my-input-topic-1");
const stream2 = kafkaStreams.getKStream("my-input-topic-2");
const stream3 = kafkaStreams.getKStream("my-input-topic-3");
// merge will make sure any message that is consumed on any of the streams
// will end up being emitted in the merged stream
// checkout other operations: join (outer, left, inner), combine, zip
// for other merge options
const mergedStream = stream1.merge(stream2).merge(stream3);
// await for 3 kafka consumer
// and 1 kafka producer to be ready
await Promise.all([
stream1.start(),
stream2.start(),
stream3.start(),
mergedStream.to("my-merged-output-topic") // BE AWARE that .to()s on a merged stream are async
]);
setTimeout(() => {
kafkaStreams.closeAll();
}, 600 * 1000);
Messages of stream "my-input-topic-1" are being written to "my-input-topic-1" and this happens for every input stream. my-merged-output-topic is empty. I have the latest version of the package and I am using Node v16.15.0