conduit icon indicating copy to clipboard operation
conduit copied to clipboard
verified publisher icon ConduitIO

Schema support

Open hariso opened this issue 1 year ago • 1 comments

Feature description

Adds support for schemas in Conduit. The work is split as in the lists below. It would be good if we start with #1561 and the conduit-commons issues (minus schema auto-generation) so we can parallelize the rest of the work.

Conduit

  • [x] #1561
  • [x] #1562
  • [ ] #1563
  • [ ] #1653
  • [ ] #1660

conduit-commons

  • [x] https://github.com/ConduitIO/conduit-commons/issues/50
  • [ ] https://github.com/ConduitIO/conduit-commons/issues/52
  • [ ] https://github.com/ConduitIO/conduit-commons/issues/53
  • [ ] https://github.com/ConduitIO/conduit-commons/issues/51

Connector SDK

  • [ ] https://github.com/ConduitIO/conduit-connector-sdk/issues/132
  • [ ] https://github.com/ConduitIO/conduit-connector-sdk/issues/137
  • [ ] https://github.com/ConduitIO/conduit-connector-sdk/issues/138

Processor SDK

  • [ ] https://github.com/ConduitIO/conduit-processor-sdk/issues/45

hariso avatar May 14 '24 14:05 hariso

A lot, if not all of the code, needed for auto-generating Avro schemas can be found here.

The utilities mentioned in both SDKs can be placed in conduit-commons, so we can re-use it.

lovromazgon avatar May 14 '24 15:05 lovromazgon

Shipped in https://github.com/ConduitIO/conduit/releases/tag/v0.11.0

lovromazgon avatar Aug 07 '24 15:08 lovromazgon