Improvement on copy events feature from a topic to another

[fabiofilz]

HI

I would like to check the possibility of when copying events from one topic to another if we could:

1 - Select which event we want to copy or select ALL.

2 - When copying the event succeeded, check if Schema exists associated with the destination topic and create it if don't exist. We had to create the schema manually/subject to the destination topic in order to not break any Kafka Connector we had in place.

This is a really good feature to copy events from DQL to the main topic to reprocess the events.

Thank you. Fabio

[tchiotludo]

Hi,

For the 1, seems to be a good idea.

For the 2, I don't understand why it will break things, avro schema are encoded in the message with an id and don't depend from topic, it should not break anything to copy a message from a topic to another topic on the same cluster :thinking:

[fabiofilz]

Hi,

Thank you for your quick response :-)

Let me send you an example:

We copied the entries from the DLQ topic to the RAW topic and this made the Connector throws errors.

To fix it we created a new schema in the RAW topic by copying the text from the DLQ schema ID 350. After saving it, it creates a new subject pointing to the same schema ID source (ID 350).

My understanding is that the schema key 350 is pointing to 2 subjects and each subject points to a topic (DLQ and RAW).

After that, we restarted the connectors and everything works fine now.

Thank you. Fabio