kafka-connect-cosmosdb icon indicating copy to clipboard operation
kafka-connect-cosmosdb copied to clipboard

Allow mapping multiple topics to single container in sink connector

Open benjaminrhope opened this issue 3 years ago • 1 comments

## Problem Statement I want to map multiple topics to one single cosmos db container like this:

"connect.cosmos.containers.topicmap": "topicA#container,topicB#container"

This is currenltly not supported since TopicContainerMap uses the BidiMap data structure, which only allows for a 1-1 relationships.

## Proposed Solution Allow multiple topics to map to one single container by allowing many-to-one relationships in TopicContainerMap.

Next Steps

  • [ ] Team consensus to proceed
  • [ ] Schedule Design Session
  • [ ] Complete Design Review

benjaminrhope avatar Jan 21 '22 12:01 benjaminrhope

This should be easy to implement. Take the provide a config for topicMap, and parse the string on topic#container. Need to consider what to do if - a) that config is not provided, b) does this make another config redundant and therefore a breaking change c) what to do if the string is supplied in the incorrect format.

ryancrawcour avatar Jan 23 '22 22:01 ryancrawcour