kafka-connect-cosmosdb
kafka-connect-cosmosdb copied to clipboard
Allow mapping multiple topics to single container in sink connector
## Problem Statement I want to map multiple topics to one single cosmos db container like this:
"connect.cosmos.containers.topicmap": "topicA#container,topicB#container"
This is currenltly not supported since TopicContainerMap
uses the BidiMap
data structure, which only allows for a 1-1 relationships.
## Proposed Solution
Allow multiple topics to map to one single container by allowing many-to-one relationships in TopicContainerMap
.
Next Steps
- [ ] Team consensus to proceed
- [ ] Schedule Design Session
- [ ] Complete Design Review
This should be easy to implement. Take the provide a config for topicMap, and parse the string on topic#container. Need to consider what to do if - a) that config is not provided, b) does this make another config redundant and therefore a breaking change c) what to do if the string is supplied in the incorrect format.