hadoop icon indicating copy to clipboard operation
hadoop copied to clipboard

HDFS-17397. Choose another DN as soon as possible, when encountering network issues

Open xleoken opened this issue 1 year ago • 27 comments

Description of PR

When there is a network issue between the client and DN, the write process will enter hang state. We hope to choose another DN as soon as possible when encountering network problems.

image

image

How was this patch tested?

Local tested.

For code changes:

  • [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • [ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

xleoken avatar Feb 26 '24 09:02 xleoken

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 34s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 43m 40s trunk passed
+1 :green_heart: compile 1m 0s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: compile 0m 55s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: checkstyle 0m 34s trunk passed
+1 :green_heart: mvnsite 0m 59s trunk passed
+1 :green_heart: javadoc 0m 50s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 42s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 :x: spotbugs 2m 37s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 :green_heart: shadedclient 34m 19s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 48s the patch passed
+1 :green_heart: compile 0m 51s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javac 0m 51s the patch passed
+1 :green_heart: compile 0m 45s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: javac 0m 45s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 21s the patch passed
+1 :green_heart: mvnsite 0m 49s the patch passed
+1 :green_heart: javadoc 0m 34s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 34s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: spotbugs 2m 33s the patch passed
+1 :green_heart: shadedclient 34m 11s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 27s hadoop-hdfs-client in the patch passed.
+1 :green_heart: asflicense 0m 37s The patch does not generate ASF License warnings.
133m 6s
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/1/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6591
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 8fa7151b2118 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 3a3d077ce1c2d68983743f0bc99ff5de79a2169c
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/1/testReport/
Max. process+thread count 668 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Feb 26 '24 12:02 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 33s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 44m 32s trunk passed
+1 :green_heart: compile 1m 1s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: compile 0m 55s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: checkstyle 0m 35s trunk passed
+1 :green_heart: mvnsite 0m 58s trunk passed
+1 :green_heart: javadoc 0m 50s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 44s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 :x: spotbugs 2m 36s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 :green_heart: shadedclient 34m 26s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 47s the patch passed
+1 :green_heart: compile 0m 52s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javac 0m 52s the patch passed
+1 :green_heart: compile 0m 46s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: javac 0m 46s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 20s the patch passed
+1 :green_heart: mvnsite 0m 50s the patch passed
+1 :green_heart: javadoc 0m 35s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 35s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: spotbugs 2m 36s the patch passed
+1 :green_heart: shadedclient 34m 16s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 27s hadoop-hdfs-client in the patch passed.
+1 :green_heart: asflicense 0m 38s The patch does not generate ASF License warnings.
134m 0s
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/2/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6591
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 2b3d0402a623 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / a05d1d4043d42c7f08885245948518477c914310
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/2/testReport/
Max. process+thread count 554 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/2/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Feb 27 '24 04:02 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 33s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 45m 48s trunk passed
+1 :green_heart: compile 1m 2s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: compile 0m 53s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: checkstyle 0m 32s trunk passed
+1 :green_heart: mvnsite 0m 55s trunk passed
+1 :green_heart: javadoc 0m 46s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 40s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 :x: spotbugs 2m 40s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 :green_heart: shadedclient 36m 52s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 49s the patch passed
+1 :green_heart: compile 0m 53s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javac 0m 53s the patch passed
+1 :green_heart: compile 0m 48s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: javac 0m 48s the patch passed
+1 :green_heart: blanks 0m 1s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 20s the patch passed
+1 :green_heart: mvnsite 0m 51s the patch passed
+1 :green_heart: javadoc 0m 34s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 33s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: spotbugs 2m 47s hadoop-hdfs-project/hadoop-hdfs-client generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1)
+1 :green_heart: shadedclient 37m 52s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 24s hadoop-hdfs-client in the patch passed.
+1 :green_heart: asflicense 0m 35s The patch does not generate ASF License warnings.
141m 15s
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/3/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6591
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 74ab123d82e3 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 9e7890090155357347b1138310a5263525390432
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/3/testReport/
Max. process+thread count 552 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/3/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Feb 27 '24 07:02 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 34s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 44m 58s trunk passed
+1 :green_heart: compile 1m 0s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: compile 0m 50s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: checkstyle 0m 32s trunk passed
+1 :green_heart: mvnsite 1m 0s trunk passed
+1 :green_heart: javadoc 0m 49s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 0m 42s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 :x: spotbugs 2m 38s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 :green_heart: shadedclient 36m 51s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 47s the patch passed
+1 :green_heart: compile 0m 52s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javac 0m 52s the patch passed
+1 :green_heart: compile 0m 44s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: javac 0m 44s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 20s the patch passed
+1 :green_heart: mvnsite 0m 49s the patch passed
+1 :green_heart: javadoc 0m 33s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 0m 35s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: spotbugs 2m 33s the patch passed
+1 :green_heart: shadedclient 34m 29s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 26s hadoop-hdfs-client in the patch passed.
+1 :green_heart: asflicense 0m 39s The patch does not generate ASF License warnings.
137m 5s
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/4/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6591
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 90820b2abf6e 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 3d0be40a300758d54ab025976e8d5ab3e7c32632
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/4/testReport/
Max. process+thread count 733 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/4/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Feb 27 '24 09:02 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 47s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 48m 53s trunk passed
+1 :green_heart: compile 1m 5s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: compile 0m 53s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: checkstyle 0m 29s trunk passed
+1 :green_heart: mvnsite 0m 56s trunk passed
+1 :green_heart: javadoc 0m 48s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 39s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 :x: spotbugs 2m 40s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 :green_heart: shadedclient 39m 30s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 49s the patch passed
+1 :green_heart: compile 0m 57s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javac 0m 57s the patch passed
+1 :green_heart: compile 0m 46s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: javac 0m 46s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 21s the patch passed
+1 :green_heart: mvnsite 0m 49s the patch passed
+1 :green_heart: javadoc 0m 35s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 32s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: spotbugs 2m 40s the patch passed
+1 :green_heart: shadedclient 40m 43s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 20s hadoop-hdfs-client in the patch passed.
+1 :green_heart: asflicense 0m 36s The patch does not generate ASF License warnings.
149m 53s
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/5/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6591
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux ed8f4f65ba3c 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 3d0be40a300758d54ab025976e8d5ab3e7c32632
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/5/testReport/
Max. process+thread count 528 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/5/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Feb 27 '24 10:02 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 32s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 44m 32s trunk passed
+1 :green_heart: compile 1m 1s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: compile 0m 56s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: checkstyle 0m 34s trunk passed
+1 :green_heart: mvnsite 0m 58s trunk passed
+1 :green_heart: javadoc 0m 51s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 43s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 :x: spotbugs 2m 37s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 :green_heart: shadedclient 34m 40s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 51s the patch passed
+1 :green_heart: compile 0m 52s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javac 0m 52s the patch passed
+1 :green_heart: compile 0m 45s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: javac 0m 45s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 20s the patch passed
+1 :green_heart: mvnsite 0m 49s the patch passed
+1 :green_heart: javadoc 0m 35s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 35s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: spotbugs 2m 34s the patch passed
+1 :green_heart: shadedclient 34m 18s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 24s hadoop-hdfs-client in the patch passed.
+1 :green_heart: asflicense 0m 38s The patch does not generate ASF License warnings.
134m 15s
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/6/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6591
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux d9b70469beb2 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 43c37ff59e4a5e323e07dc4c9a43ba0f6e85bce4
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/6/testReport/
Max. process+thread count 729 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/6/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Feb 28 '24 03:02 hadoop-yetus

hi, @ZanderXu please take a review, thanks

xleoken avatar Feb 28 '24 09:02 xleoken

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 33s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 43m 54s trunk passed
+1 :green_heart: compile 1m 0s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: compile 0m 55s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: checkstyle 0m 32s trunk passed
+1 :green_heart: mvnsite 1m 0s trunk passed
+1 :green_heart: javadoc 0m 51s trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 40s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 :x: spotbugs 2m 37s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 :green_heart: shadedclient 34m 18s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 47s the patch passed
+1 :green_heart: compile 0m 51s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javac 0m 51s the patch passed
+1 :green_heart: compile 0m 45s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: javac 0m 45s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 21s the patch passed
+1 :green_heart: mvnsite 0m 48s the patch passed
+1 :green_heart: javadoc 0m 34s the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 :green_heart: javadoc 0m 34s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: spotbugs 2m 33s the patch passed
+1 :green_heart: shadedclient 34m 16s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 25s hadoop-hdfs-client in the patch passed.
+1 :green_heart: asflicense 0m 37s The patch does not generate ASF License warnings.
133m 8s
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/7/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6591
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux e7615f6b0cb7 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 06bfc72c2b4fed56f3b75cda94491ca31afe9fbb
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/7/testReport/
Max. process+thread count 552 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/7/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Feb 28 '24 10:02 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 33s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 45m 6s trunk passed
+1 :green_heart: compile 1m 0s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: compile 0m 55s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: checkstyle 0m 31s trunk passed
+1 :green_heart: mvnsite 1m 1s trunk passed
+1 :green_heart: javadoc 0m 50s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 0m 42s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 :x: spotbugs 2m 46s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 :green_heart: shadedclient 36m 24s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 52s the patch passed
+1 :green_heart: compile 0m 55s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javac 0m 55s the patch passed
+1 :green_heart: compile 0m 46s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: javac 0m 46s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 21s the patch passed
+1 :green_heart: mvnsite 0m 49s the patch passed
+1 :green_heart: javadoc 0m 36s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 0m 34s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: spotbugs 2m 43s the patch passed
+1 :green_heart: shadedclient 38m 23s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 28s hadoop-hdfs-client in the patch passed.
+1 :green_heart: asflicense 0m 37s The patch does not generate ASF License warnings.
141m 31s
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/8/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6591
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux e369432213b5 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / cfef0ddde37ce9fb94e2ad356615c7df20f90cc3
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/8/testReport/
Max. process+thread count 553 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/8/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Mar 05 '24 11:03 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 32s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 47m 1s trunk passed
+1 :green_heart: compile 1m 0s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: compile 0m 55s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: checkstyle 0m 34s trunk passed
+1 :green_heart: mvnsite 0m 58s trunk passed
+1 :green_heart: javadoc 0m 49s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 0m 44s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 :x: spotbugs 2m 38s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 :green_heart: shadedclient 34m 12s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 49s the patch passed
+1 :green_heart: compile 0m 51s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javac 0m 51s the patch passed
+1 :green_heart: compile 0m 45s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: javac 0m 45s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 20s the patch passed
+1 :green_heart: mvnsite 0m 47s the patch passed
+1 :green_heart: javadoc 0m 34s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 0m 34s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: spotbugs 2m 33s the patch passed
+1 :green_heart: shadedclient 34m 30s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 27s hadoop-hdfs-client in the patch passed.
+1 :green_heart: asflicense 0m 37s The patch does not generate ASF License warnings.
136m 36s
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/9/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6591
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 48f8ed5137d7 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 6dfaaef8c282276ac4e4f88468644f11cd67a11d
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/9/testReport/
Max. process+thread count 705 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/9/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Mar 06 '24 03:03 hadoop-yetus

cc @Hexiaoqiao please take a review, thanks

xleoken avatar Mar 10 '24 11:03 xleoken

cc @steveloughran PTAL

xleoken avatar Mar 13 '24 07:03 xleoken

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 34s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 46m 45s trunk passed
+1 :green_heart: compile 1m 0s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: compile 0m 53s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: checkstyle 0m 31s trunk passed
+1 :green_heart: mvnsite 0m 59s trunk passed
+1 :green_heart: javadoc 0m 48s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 0m 40s trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 :x: spotbugs 2m 45s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 :green_heart: shadedclient 38m 43s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 51s the patch passed
+1 :green_heart: compile 0m 53s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javac 0m 53s the patch passed
+1 :green_heart: compile 0m 46s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: javac 0m 46s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 19s the patch passed
+1 :green_heart: mvnsite 0m 49s the patch passed
+1 :green_heart: javadoc 0m 35s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 0m 34s the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 :green_heart: spotbugs 2m 46s the patch passed
+1 :green_heart: shadedclient 38m 40s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 25s hadoop-hdfs-client in the patch passed.
+1 :green_heart: asflicense 0m 36s The patch does not generate ASF License warnings.
145m 1s
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/10/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6591
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 608726f0bdb6 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 468b83e2b279da3d328537199adbd064bce54b16
Default Java Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/10/testReport/
Max. process+thread count 552 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/10/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Mar 13 '24 09:03 hadoop-yetus

I don't go near HDFS code; HDFS dev team are happy about this. sorry.

steveloughran avatar Mar 13 '24 12:03 steveloughran

I don't go near HDFS code; HDFS dev team are happy about this. sorry.

OK

xleoken avatar Mar 14 '24 00:03 xleoken

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 29s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 42m 57s trunk passed
+1 :green_heart: compile 1m 1s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: compile 0m 55s trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: checkstyle 0m 35s trunk passed
+1 :green_heart: mvnsite 0m 59s trunk passed
+1 :green_heart: javadoc 0m 51s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 0m 43s trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
-1 :x: spotbugs 2m 35s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 :green_heart: shadedclient 35m 0s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 51s the patch passed
+1 :green_heart: compile 0m 51s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javac 0m 51s the patch passed
+1 :green_heart: compile 0m 45s the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: javac 0m 45s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 21s the patch passed
+1 :green_heart: mvnsite 0m 49s the patch passed
+1 :green_heart: javadoc 0m 34s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 0m 35s the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: spotbugs 2m 33s the patch passed
+1 :green_heart: shadedclient 34m 44s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 27s hadoop-hdfs-client in the patch passed.
+1 :green_heart: asflicense 0m 37s The patch does not generate ASF License warnings.
133m 53s
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/11/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6591
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux aaf0a3ef1b26 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 73d6c12734975d4adbd52f39e810478322b55a9f
Default Java Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/11/testReport/
Max. process+thread count 699 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/11/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Mar 21 '24 10:03 hadoop-yetus

@xleoken Thanks for your proposal. I am not sure this is the proper solution for your case as @ZanderXu mentioned. IIUC, you expect to fast fail when meet network issue between client and the first DataNode while write data to pipeline, right? IMO, it is difficult to determine to do that because, a. Sometimes we could not determine that it is client to the first DataNode network issue only using the time cost of ACK. Timeout between DataNodes in pipeline could also lead client wait time out IMO. b. If fast fail and recovery pipeline, the time cost could be more considerable, such as re-create pipeline and transfer data will involve more time cost when have writen out more than 10MB.

For this case, we have discussed times, I think we need to split it to two step, report metrics back to client, then improve strategy (fast fail or switch dn or some other way based on different metrics). FYI. Thanks again.

Hexiaoqiao avatar Mar 25 '24 09:03 Hexiaoqiao

@Hexiaoqiao thanks for your review, you're right, I'll think about it again.

xleoken avatar Mar 26 '24 06:03 xleoken

@Hexiaoqiao I tried to learn more about the DataStreamer.

1、Write image

2、Default block size: 128MB, packet size: 64KB, Add debug log, DataStreamer -> ResponseProcessor image

3、Simulate error exception image

4、dfs.datanode.socket.write.timeout and dfs.client.slow.io.warning.threshold.ms image

xleoken avatar Mar 28 '24 07:03 xleoken

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 31s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 44m 30s trunk passed
+1 :green_heart: compile 1m 1s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: compile 0m 57s trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: checkstyle 0m 34s trunk passed
+1 :green_heart: mvnsite 0m 59s trunk passed
+1 :green_heart: javadoc 0m 50s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 0m 44s trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
-1 :x: spotbugs 2m 38s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 :green_heart: shadedclient 34m 49s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 49s the patch passed
+1 :green_heart: compile 0m 53s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javac 0m 53s the patch passed
+1 :green_heart: compile 0m 45s the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: javac 0m 45s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 21s the patch passed
+1 :green_heart: mvnsite 0m 47s the patch passed
+1 :green_heart: javadoc 0m 36s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 0m 35s the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: spotbugs 2m 34s the patch passed
+1 :green_heart: shadedclient 34m 38s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 25s hadoop-hdfs-client in the patch passed.
+1 :green_heart: asflicense 0m 37s The patch does not generate ASF License warnings.
135m 1s
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/12/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6591
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 8e980caff1e4 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 73d6c12734975d4adbd52f39e810478322b55a9f
Default Java Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/12/testReport/
Max. process+thread count 552 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/12/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Mar 28 '24 09:03 hadoop-yetus

cc @Hexiaoqiao @ZanderXu @slfan1989 @shahrs87

xleoken avatar Apr 08 '24 07:04 xleoken

@xleoken Thanks for your works and patch. However I don't think it is the graceful solution or improvement as mentioned above. Thanks again.

Hexiaoqiao avatar Apr 08 '24 07:04 Hexiaoqiao

This patch works for my case, lgtm.

charlesy6 avatar Apr 09 '24 01:04 charlesy6

Thanks!

On Thu, Mar 28, 2024 at 4:17 PM Apache Hadoop Yetus Account < @.***> wrote:

💔 -1 overall Vote Subsystem Runtime Logfile Comment +0 🆗 reexec 0m 31s Docker mode activated. _ Prechecks _ +1 💚 dupname 0m 0s No case conflicting files found. +0 🆗 codespell 0m 1s codespell was not available. +0 🆗 detsecrets 0m 1s detect-secrets was not available. +1 💚 @author https://github.com/author 0m 0s The patch does not contain any @author https://github.com/author tags. -1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. _ trunk Compile Tests _ +1 💚 mvninstall 44m 30s trunk passed +1 💚 compile 1m 1s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 +1 💚 compile 0m 57s trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 +1 💚 checkstyle 0m 34s trunk passed +1 💚 mvnsite 0m 59s trunk passed +1 💚 javadoc 0m 50s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 +1 💚 javadoc 0m 44s trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 -1 ❌ spotbugs 2m 38s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/12/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings. +1 💚 shadedclient 34m 49s branch has no errors when building and testing our client artifacts. _ Patch Compile Tests _ +1 💚 mvninstall 0m 49s the patch passed +1 💚 compile 0m 53s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 +1 💚 javac 0m 53s the patch passed +1 💚 compile 0m 45s the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 +1 💚 javac 0m 45s the patch passed +1 💚 blanks 0m 0s The patch has no blanks issues. +1 💚 checkstyle 0m 21s the patch passed +1 💚 mvnsite 0m 47s the patch passed +1 💚 javadoc 0m 36s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 +1 💚 javadoc 0m 35s the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 +1 💚 spotbugs 2m 34s the patch passed +1 💚 shadedclient 34m 38s patch has no errors when building and testing our client artifacts. _ Other Tests _ +1 💚 unit 2m 25s hadoop-hdfs-client in the patch passed. +1 💚 asflicense 0m 37s The patch does not generate ASF License warnings. 135m 1s Subsystem Report/Notes Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/12/artifact/out/Dockerfile GITHUB PR #6591 https://github.com/apache/hadoop/pull/6591 Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets uname Linux 8e980caff1e4 5.15.0-94-generic #104 https://github.com/apache/hadoop/pull/104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality dev-support/bin/hadoop.sh git revision trunk / 73d6c12 https://github.com/apache/hadoop/commit/73d6c12734975d4adbd52f39e810478322b55a9f Default Java Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/12/testReport/ Max. process+thread count 552 (vs. ulimit of 5500) modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6591/12/console versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2 Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

— Reply to this email directly, view it on GitHub https://github.com/apache/hadoop/pull/6591#issuecomment-2024733961, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGHNPXUIDEMCB2NKM337C2TY2PGYJAVCNFSM6AAAAABDZ3KKAGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRUG4ZTGOJWGE . You are receiving this because you are subscribed to this thread.Message ID: @.***>

tnsbn avatar Apr 12 '24 03:04 tnsbn

lgtm

kristgpt avatar Apr 13 '24 08:04 kristgpt

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
_ Prechecks _
+1 :green_heart: dupname 0m 00s No case conflicting files found.
+0 :ok: spotbugs 0m 01s spotbugs executables are not available.
+0 :ok: codespell 0m 01s codespell was not available.
+0 :ok: detsecrets 0m 01s detect-secrets was not available.
+1 :green_heart: @author 0m 00s The patch does not contain any @author tags.
-1 :x: test4tests 0m 00s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 92m 28s trunk passed
+1 :green_heart: compile 5m 11s trunk passed
+1 :green_heart: checkstyle 4m 45s trunk passed
+1 :green_heart: mvnsite 5m 45s trunk passed
+1 :green_heart: javadoc 4m 58s trunk passed
+1 :green_heart: shadedclient 148m 52s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 3m 28s the patch passed
+1 :green_heart: compile 2m 48s the patch passed
+1 :green_heart: javac 2m 48s the patch passed
+1 :green_heart: blanks 0m 00s The patch has no blanks issues.
+1 :green_heart: checkstyle 2m 06s the patch passed
+1 :green_heart: mvnsite 3m 01s the patch passed
+1 :green_heart: javadoc 2m 27s the patch passed
+1 :green_heart: shadedclient 161m 28s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: asflicense 5m 27s The patch does not generate ASF License warnings.
426m 48s
Subsystem Report/Notes
GITHUB PR https://github.com/apache/hadoop/pull/6591
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname MINGW64_NT-10.0-17763 12bdaf86eb0b 3.4.10-87d57229.x86_64 2024-02-14 20:17 UTC x86_64 Msys
Build tool maven
Personality /c/hadoop/dev-support/bin/hadoop.sh
git revision trunk / 73d6c12734975d4adbd52f39e810478322b55a9f
Default Java Azul Systems, Inc.-1.8.0_332-b09
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6591/1/testReport/
modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6591/1/console
versions git=2.44.0.windows.1
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Apr 25 '24 03:04 hadoop-yetus

LGTM

daragu avatar May 13 '24 01:05 daragu