snowflake-kafka-connector
snowflake-kafka-connector copied to clipboard
Support multiple topics to one table in snowflake.topic2table.map
Currently, part of the validation of the snowflake.topic2table.map parameter requires all of the tables to be unique so multiple topics cannot be written to the same table
"topics": "topic1,topic2",
"snowflake.topic2table.map": "topic1:table1,topic2:table2",
We have multiple topics with the same data format though, and would like to be able to write them to the same table to reduce the special processing we need to do on the snowflake side
"topics": "topic1,topic2",
"snowflake.topic2table.map": "topic1:table1,topic2:table1",
I built a JAR with the check stopping this removed and ran it for about a day and the combined table got the same data as the three individual ones union'd together. I would like to make a PR for my branch but it doesn't look like I have permission to push it to origin.
Looks like no movement on this? This should definitely be supported.
I'd also like to use this feature.
Has there been any update on this feature?
+1 for wanting support for this feature
+1
This should be supported now as part of 1.8.1 release, please give it a try!