[Feature][Spark Starter] Add suuport for Spark Operator

[Rianico]

Search before asking

• [X] I had searched in the feature and found no similar feature requirement.

Description

Recently, we tried to run Seatunnel on our k8s cluster, and found that seatunnel had supported flink operator and seatunnel engine but not spark operator. I think adding support for spark operator could be a common requirement.

Usage Scenario

Run spark task on k8s cluster in Spark Operator way.

Related issues

No response

Are you willing to submit a PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[Rianico]

If this feature is approved, it can be assigned to me.

[liunaijie]

you can deploy the cluster and setup spark home to spark lib dir or mount spark lib to /opt/spark then you can submit to spark on k8s.

[Rianico]

you can deploy the cluster and setup spark home to spark lib dir or mount spark lib to /opt/spark then you can submit to spark on k8s.

However, relying solely on native spakr-submit is not advisable. It only creates the naked pods, which is not recommended by k8s official, and lacks some essential components such as services and ingresses. It might not suffice for production scenarios.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.