[Feature] add dockerfile

[liunaijie]

Purpose of this pull request

close https://github.com/apache/seatunnel/issues/5475

add dockerfile, added docker, docker-full maven profile. when use docker profile it will build the light docker image (only include fake, console connector) (image name will be apache/seatunnel:<VERSION>) when use docker-full profile it will build the full connector docker image (image name will be apache/seatunnel:<VERSION>-full)

Now it need manually publish docker image to docker hub.

Does this PR introduce any user-facing change?

No

How was this patch tested?

with github action, added a new job, will run docker image and submit a job.

Check list

• [ ] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
• [x] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
• [ ] If you are contributing the connector code, please check that the following files are updated:
  1. Update change log that in connector document. For more details you can refer to connector-v2
  2. Update plugin-mapping.properties and add new connector information in it
  3. Update the pom file of seatunnel-dist
• [ ] Update the release-note.

[Hisoka-X]

Could you retrigger there failed CI? Thanks.

[liunaijie]

Could you retrigger there failed CI? Thanks.

me? I can't trigger the CI in this repo

[Hisoka-X]

Our CI runnning on your own fork repository, so you can retirgger it in your repository action.

[liunaijie]

Our CI runnning on your own fork repository, so you can retirgger it in your repository action.

Done.

[liunaijie]

@ruanwenjun @Hisoka-X PTAL

[liunaijie]

This document only for SeaTunnel Zeta and is more appropriately named docker_for_zeta.md

Done

[liunaijie]

@FlechazoW i update the document add flink/spark engine. PTAL.

[FlechazoW]

You need to enable ci/cd first.

[Hisoka-X]

Same as https://github.com/apache/seatunnel/pull/5594#issuecomment-1757087541 .

[liunaijie]

@Hisoka-X the test result attached,

[Hisoka-X]

Hi @liunaijie . Could you follow #5594 to add a test case for docker?

[liunaijie]

Hi @liunaijie . Could you follow #5594 to add a test case for docker?

I add a test case on maven verify phase. run a command to verify.

[liunaijie]

One question: when i pakcage locally, i find the starter jar has version info. like seatunnel-starter-2.3.3-SNAPSHOT.jar not the format seatunnel-starter.jar. so when i submit job, it will get error. I manually change the seatunnel.sh script to pass the test in local. Does anyone know why the finnalName is without version but the jar still has version info.

[Hisoka-X]

One question: when i pakcage locally, i find the starter jar has version info. like seatunnel-starter-2.3.3-SNAPSHOT.jar not the format seatunnel-starter.jar. so when i submit job, it will get error. I manually change the seatunnel.sh script to pass the test in local. Does anyone know why the finnalName is without version but the jar still has version info.

In my environment, these are not exist seatunnel-starter-2.3.3-SNAPSHOT.jar but only seatunnel-starter.jar. Where did you get seatunnel-starter-2.3.3-SNAPSHOT.jar jar?

[liunaijie]

One question: when i pakcage locally, i find the starter jar has version info. like seatunnel-starter-2.3.3-SNAPSHOT.jar not the format seatunnel-starter.jar. so when i submit job, it will get error. I manually change the seatunnel.sh script to pass the test in local. Does anyone know why the finnalName is without version but the jar still has version info.

In my environment, these are not exist seatunnel-starter-2.3.3-SNAPSHOT.jar but only seatunnel-starter.jar. Where did you get seatunnel-starter-2.3.3-SNAPSHOT.jar jar?

package dist module, unzip apache-seatunnel-2.3.3-SNAPSHOT-bin.tar.gz.

[Hisoka-X]

Fine for me too.

[liunaijie]

Fine for me too.

It work well in CI phase https://github.com/liunaijie/seatunnel/actions/runs/6785810765/job/18475827132, maybe i have something change in my local i am not noticed.

[liunaijie]

Fine for me too.

It work well in CI phase https://github.com/liunaijie/seatunnel/actions/runs/6785810765/job/18475827132, maybe i have something change in my local i am not noticed.

I re-run clean, install command in version 2.3.4-snapshot. it works for me.

[liunaijie]

@Hisoka-X @ruanwenjun @EricJoy2048 PTAL when you have time. when we push tag, it will build, verify, push docker image to docker hub. Here is a successful job link for your refer: https://github.com/liunaijie/seatunnel/actions/runs/6811013504/job/18520474386. and docker hub images generate by this job: https://hub.docker.com/r/liunaijie/seatunnel/tags

[liunaijie]

removed action file, now it won't auto publish image to docker hub. when the release manager want publish image, he can package dist module with docker profile, the image will auto build, need manually publish.

[Hisoka-X]

cc @TyrantLucifer

[EricJoy2048]

@TyrantLucifer PTAL

[liunaijie]

@EricJoy2048 @TyrantLucifer Do we need full connector docker image? And I still want add github action to auto deploy docker image to docker hub. For the case @TyrantLucifer mentioned, we will rollback/fix when review process find some bug.

1. I am not clear about the pre-release process. but i think we can only monitor the tag/branch of finial tag/branch instand of pre-release tag/branch.
2. or we can delete the error image manually when we find bug

[liunaijie]

@Hisoka-X @hailin0 can you help take a look about this pr. add Dockerfile, we can build publish docker image to offical docker hub. user won't need build the image byself. and we also can include all lib to image user can skip the install plugin step.