[Feature][Web Server] Add SeaTunnel Job Executor

[EricJoy2048]

Search before asking

• [X] I had searched in the feature and found no similar feature requirement.

Description

SeaTunnel Job Executor is an executor tool, It is designed for scheduling tools to execute SeaTunnel jobs which is manager in SeaTunnel Web. It need some key features.

1. Executor can run by shell, It needs to provide a shell script as an entry for execution. For example sh seatunnel-executor.sh -i st_job_id1
2. Executor have an argument named --jobId(-i), the jobId is required. The jobId is the the job's id in the SeaTunnel Web.

How SeaTunnel Executor work?

1. Call the SeaTunnel Web API and get the job information of the jobId. The job information include job config file engine(SeaTunnel Engine/Spark/Flink) Deploy Mode and other we need.
2. Like start-seatunnel-spark-connector-v2.sh start-seatunnel-flink-connetor-v2.sh seatunnel.sh to submit the job.
3. Monitor the job status and call SeaTunnel Web API to update the job's status.
4. Output the job client logs, because the scheduler system need logs.

Welcome to supplement

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[TyrantLucifer]

@lhboss

link to #2666

[skyoct]

@EricJoy2048 I want to try it. Please give it to me. thx

[TyrantLucifer]

@EricJoy2048 I want to try it. Please give it to me. thx

Hi @skyoct , this task has assigned to @lhboss, could you please try other tasks?

[skyoct]

@EricJoy2048 I want to try it. Please give it to me. thx

Hi @skyoct , this task has assigned to @lhboss, could you please try other tasks?

ok, thx

[cason0126]

Does job information including jvm resource parameters like -Xms -Xmx？If Job Information includes these paras ，Whether scripts in #3155 #3307 should accept these parameters ？ @EricJoy2048

[EricJoy2048]

@TyrantLucifer Here are some interaction processes between the Executor and the Scheduler System and SeaTunnel Web APIs.

[EricJoy2048]

Does job information including jvm resource parameters like -Xms -Xmx？If Job Information includes these paras ，Whether scripts in #3155 #3307 should accept these parameters ？ @EricJoy2048

All engine resources required for job running will be defined in jobfile#env. The jvm parameters of Executor will config in Executor shell named seatunnel-executor.sh.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[TyrantLucifer]

This task had been done, closed.