seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Feature][Zeta] Add Zeta Http Client

Open EricJoy2048 opened this issue 2 years ago • 3 comments

Search before asking

  • [X] I had searched in the feature and found no similar feature requirement.

Description

To use SeaTunnel, the current user first needs to create and write a configuration file that specifies the engine to run the job and the parameters associated with the engine. Then define the "source," "transform," and "sink" of the job. We want to provide a client that allows the user to define the engine, source, and transformation, and receive information for a job directly from the code in the client, rather than starting with a configuration file. Users can then submit job definition information through the client, and SeaTunnel will run these jobs. After a job is submitted, the user can obtain the running status of the job through the client. Users can use this client to manage jobs that are already running, such as stop jobs, temporary jobs, and so on.

Usage Scenario

Motivation

The solution we adopted is to design Seatunnel Http Client instead of Seatnnel CLI. Compared with Seatunnel CLI, Seatunnel Http Client has the following advantages:

The REST API is more lightweight than the Seatnnel CLI Unified model guided implementation of multi-language support

The REST API runs as part of JobMaster on the computing engine (Seatunnel Zeta) and is supported by the server. If we use Zeta as the computing engine, Seatunnel Http Client will communicate with the Zeta engine, which is Responsible for Zeta cluster-level operations such as submitting jobs, listing jobs, requesting cluster status, etc.

https://docs.google.com/document/d/1GYSyV6NY6DwLTWisy7VFSxFaocJgR3BFDn3v9imuWwg/edit#heading=h.pjvx8ia58ecq

Related issues

No response

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

EricJoy2048 avatar Jun 20 '23 08:06 EricJoy2048

Like Kafka Connector?,Good,It is better to package an SDK directly

wu-a-ge avatar Jun 21 '23 08:06 wu-a-ge

I am willing to submit a PR.

zhibinF avatar Jun 30 '23 09:06 zhibinF

I am willing to submit a PR.

Already assigned to you.

liugddx avatar Jun 30 '23 09:06 liugddx

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] avatar Jul 31 '23 00:07 github-actions[bot]