seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Feature][Api-draft] Create a bootstrap for connector development

Open EricJoy2048 opened this issue 2 years ago • 1 comments

Search before asking

  • [X] I had searched in the feature and found no similar feature requirement.

Description

Now we are using API draft to develop new connectors, and we have found some problems. Whether Source API or Sink API have many interface, these interface are use for different functions。For example, SinkWriter#snapshotState() is use for connectors that need to save state and reload in case of failure recovery. SinkWriter#prepareCommit() is use for connector that need to 2PC submit.

With reference to airbyte, can we develop a cli boot tool through which users can create a connector basic code? They can use --with_ state, --with_ split and other parameters specify the functions required by their connector. Based on these parameters, we automatically generate the corresponding connector code, and only show them the interfaces that need to be implemented.

I believe this will make the development of connector easier.

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

EricJoy2048 avatar Jun 16 '22 03:06 EricJoy2048

We can provide simple soure and sink, masking some advanced features.

ashulin avatar Jun 16 '22 12:06 ashulin