conduit icon indicating copy to clipboard operation
conduit copied to clipboard

Published 20 hours ago • verified publisher icon ConduitIO

OpenCDC Integration

Open lovromazgon opened this issue 1 year ago • 0 comments

Description

This PR updates Conduit to use a new Record structure that matches OpenCDC records. Apart from that it:

  • Updates versions of all builtin plugins and uses sdk.Connector to register them in the registry.
  • Changes processors insertfieldkey and insertfieldpayload by removing the config option timestamp.field (see comment).
  • Moves field Record.SourceID to Message.SourceID.
  • Creates an intermediary record that is passed to the JS processor instead of our internal record. This allows the JS processor to manipulate all fields including structured data.

Fixes #511, fixes #468.

Quick checks:

  • [X] I have followed the Code Guidelines.
  • [X] There is no other pull request for the same update/change.
  • [X] I have written unit tests.
  • [X] I have made sure that the PR is of reasonable size and can be easily reviewed.

lovromazgon avatar Aug 04 '22 18:08 lovromazgon