Joining Kafka topics

[zentol]

Importing a recipe request from the Flink slack.

could also include a recipe for how to join two Kafka topics with the race conditions, competing against each other that would be very helpful

--- Mehul Batra

[MehulBatra]

if say we want to join two topics on a key and in one topic the data is coming faster than the other one, I want the first topic to wait for the data that is going to come on the second topic in case there is a delay in the data or the join could not happen, once they are joined then we should move ahead/ or forget that key.

[knaufk]

if say we want to join two topics on a key and in one topic the data is coming faster than the other one, I want the first topic to wait for the data that is going to come on the second topic in case there is a delay in the data or the join could not happen, once they are joined then we should move ahead/ or forget that key.

So, there is always exactly one record with the same key in each of the topics? Can you expand a bit on the use case and nature of those topics?

[MehulBatra]

yes, exactly one record coming as the key, as it's a primary key, UseCase: Just joining two Kafka topics, with different amounts of speed records are coming so we need to wait so that records don't pass without joining.