mongo-kafka icon indicating copy to clipboard operation
mongo-kafka copied to clipboard

Published 20 hours ago • verified publisher icon mongodb

new feature about CDC Handler for 1 => N(multi row) data handle

Open gjchoi opened this issue 3 years ago • 0 comments

I'd like to develop Custom CDC Handler for N data insert with 1 sinkData. But I can't find any base kafka connect feature about this ( kafka connect options, SMT ) And CDCHandler, CDCOperation to make custom CDC Handle below

CdcOperation

WriteModel<BsonDocument> perform(SinkDocument doc);

CdcHandler

  public abstract Optional<WriteModel<BsonDocument>> handle(SinkDocument doc);

only support 1 => 1 data handle.

so I make a PR about this new feature

  • sink connector option added ( change.data.capture.multi.row.handler )
  • CdcMultiRowOperation, CdcMultiRowHanlder added
  • modified sink task logic that return list<WriteModel> and append to bulkWrite

Can you review this PR for new feature add?

or

Is there any other way for this I don't know?

gjchoi avatar Aug 22 '22 11:08 gjchoi