flink icon indicating copy to clipboard operation
flink copied to clipboard

[FLINK-29306][shell] Fix the error detection of multiple flink-dist-*.jar for config.sh

Open lzshlzsh opened this issue 3 years ago • 3 comments

What is the purpose of the change

The following shell command always make FLINK_DIST_COUNT=1 FLINK_DIST_COUNT="$(echo "$FLINK_DIST" | wc -l)" and the following condition will always be false, so fail to check multiple flink-dist*.jar。 [[ "$FLINK_DIST_COUNT" -gt 1 ]] for example:

# FLINK_DIST=":/Users/lzs/.data/github/flink/flink-dist/target/flink-1.17-SNAPSHOT-bin/flink-1.17-SNAPSHOT/lib/flink-dist-1.17-SNAPSHOT.jar:/Users/lzs/.data/github/flink/flink-dist/target/flink-1.17-SNAPSHOT-bin/flink-1.17-SNAPSHOT/lib/flink-dist-1.17-xxx.jar"
# echo "$FLINK_DIST" | wc -l
1

Brief change log

  • fix the check logic when more than one file matching flink-dist*.jar exists in /lib

Verifying this change

This change added tests and can be verified as follows:

  • create a fake flink-dist*.jar in lib, touch lib/flink-dist-fake.jar
  • run flink shell command, ./bin/flink ERROR message should print [ERROR] Multiple flink-dist*.jar found in ...

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

lzshlzsh avatar Sep 14 '22 17:09 lzshlzsh

CI report:

  • b734f68a2ce5bc5a77dade6ed426e0bceb528f1e Azure: SUCCESS
Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

flinkbot avatar Sep 14 '22 17:09 flinkbot

@flinkbot run azure

lzshlzsh avatar Sep 15 '22 14:09 lzshlzsh

Good spot. I've tested the change locally, by calling the function changes manually, and it appears to be working well now.

I have added my approval, however there is a build failure that may need investigating first.

It seems that there is a network error that causes the ci test to fail. I will run it again.

2022-09-14T18:08:17.6287381Z curl: (18) transfer closed with 314614133 bytes remaining to read
2022-09-14T18:08:18.0516809Z /home/vsts/work/1/s/flink-end-to-end-tests/test-scripts/kafka-common.sh: line 138: kill: (150980) - No such process

lzshlzsh avatar Sep 15 '22 14:09 lzshlzsh