flink icon indicating copy to clipboard operation
flink copied to clipboard

[FLINK-35625][cli] Merge "flink run" and "flink run-application" functionality, deprecate "run-application"

Open ferenc-csaky opened this issue 8 months ago • 4 comments

What is the purpose of the change

Merge the functionality of "flink run" and "flink run-application" under the "run" command to make job deployment more consistent. Also deprecate "run-application" so it can be removed together with "yarn-per-job" in the future.

Brief change log

  • Introduced isApplication(...) check method in FrontendCli to determine necessary deployment steps.
  • Handle application mode deployment in FrontendCli#run.
  • In FrontendCli#runApplication, simply call FrontendCli#run after argument check.
  • Added unit tests to cover the introduced code flow changes in CliFrontendRunTests.
  • Updated, adapted CLI help/usage messages, and relevant config option documentation.
  • Marked run-application deprecated in code, config docs, and help/usage messages.

Verifying this change

Unit tests were added to cover the introduced changes.

Also did a manual check, where I executed the following on a locally built Flink and minikube setup:

$ ./bin/flink run-application \
    --target kubernetes-application \
    -Dkubernetes.cluster-id=flink-app-cluster-1 \
    -Dkubernetes.container.image.ref=flink:1.19 \
    local:///opt/flink/examples/streaming/TopSpeedWindowing.jar

$ ./bin/flink run \
    --target kubernetes-application \
    -Dkubernetes.cluster-id=flink-app-cluster-2 \
    -Dkubernetes.container.image.ref=flink:1.19 \
    local:///opt/flink/examples/streaming/TopSpeedWindowing.jar

$ kubectl get pods
NAME                                         READY   STATUS    RESTARTS   AGE
flink-app-cluster-1-778bb58994-q8hg5         1/1     Running   0          30s
flink-app-cluster-1-taskmanager-1-1          1/1     Running   0          22s
flink-app-cluster-2-6555f5fc76-2sspj         1/1     Running   0          18s
flink-app-cluster-2-taskmanager-1-1          1/1     Running   0          9s

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

ferenc-csaky avatar Jun 17 '24 18:06 ferenc-csaky