hadoop icon indicating copy to clipboard operation
hadoop copied to clipboard

HADOOP-18382. SDK upgrade prerequisites

Open ahmarsuhail opened this issue 3 years ago • 6 comments

Description of PR

Adds in warn logs for things that will be removed in the upgrade. Also adds in documentation around the forthcoming changes.

How was this patch tested?

Tested in eu-west-1 by running mvn -Dparallel-tests -DtestsThreadCount=16 clean verify.

ITestSelectLandsat timing out, but this is a known issue.

ahmarsuhail avatar Aug 04 '22 09:08 ahmarsuhail

Thanks @monthonk!

@steveloughran how can the yetus deprecation errors be handled? Also for credential providers, do we want to allow custom V1 credential providers in delegation tokens and otherwise? They can be used with the V2 client like this (I've opened a draft PR with the first set of changes for the upgrade). I wasn't sure if we want to support them, will update the documentation I've written in this PR accordingly.

ahmarsuhail avatar Aug 05 '22 14:08 ahmarsuhail

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 48s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 1s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+0 :ok: markdownlint 0m 0s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 11 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 40m 38s trunk passed
+1 :green_heart: compile 0m 54s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: compile 0m 45s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 0m 43s trunk passed
+1 :green_heart: mvnsite 0m 53s trunk passed
+1 :green_heart: javadoc 0m 39s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 40s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 28s trunk passed
+1 :green_heart: shadedclient 23m 52s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 35s the patch passed
+1 :green_heart: compile 0m 40s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
-1 :x: javac 0m 40s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 16 new + 42 unchanged - 2 fixed = 58 total (was 44)
+1 :green_heart: compile 0m 34s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 :x: javac 0m 34s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 16 new + 41 unchanged - 2 fixed = 57 total (was 43)
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 24s the patch passed
+1 :green_heart: mvnsite 0m 37s the patch passed
+1 :green_heart: javadoc 0m 19s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 27s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 12s the patch passed
+1 :green_heart: shadedclient 23m 34s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 38s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 42s The patch does not generate ASF License warnings.
103m 55s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/6/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/4698
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname Linux 57d4c05899cc 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 5f81bceaaf0bb2b966668f3fc49c18016d4077d7
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/6/testReport/
Max. process+thread count 607 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/6/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 10 '22 11:08 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 50s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+0 :ok: markdownlint 0m 1s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 13 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 40m 58s trunk passed
+1 :green_heart: compile 0m 51s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: compile 0m 46s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 0m 44s trunk passed
+1 :green_heart: mvnsite 0m 52s trunk passed
+1 :green_heart: javadoc 0m 39s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 41s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 28s trunk passed
+1 :green_heart: shadedclient 23m 41s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 35s the patch passed
+1 :green_heart: compile 0m 41s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
-1 :x: javac 0m 41s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 1 new + 42 unchanged - 2 fixed = 43 total (was 44)
+1 :green_heart: compile 0m 33s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 :x: javac 0m 33s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 1 new + 41 unchanged - 2 fixed = 42 total (was 43)
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 24s the patch passed
+1 :green_heart: mvnsite 0m 39s the patch passed
+1 :green_heart: javadoc 0m 19s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 28s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 10s the patch passed
+1 :green_heart: shadedclient 23m 19s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 36s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 41s The patch does not generate ASF License warnings.
103m 47s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/7/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/4698
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname Linux e2ef649214b2 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / f20831ffb8132c465d51e633ea4ca886bcf31eed
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/7/testReport/
Max. process+thread count 533 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/7/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 10 '22 15:08 hadoop-yetus

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 54s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 1s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+0 :ok: markdownlint 0m 0s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 13 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 40m 55s trunk passed
+1 :green_heart: compile 0m 53s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: compile 0m 45s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 0m 43s trunk passed
+1 :green_heart: mvnsite 0m 52s trunk passed
+1 :green_heart: javadoc 0m 40s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 39s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 26s trunk passed
+1 :green_heart: shadedclient 23m 47s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 35s the patch passed
+1 :green_heart: compile 0m 40s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javac 0m 40s hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 42 unchanged - 2 fixed = 42 total (was 44)
+1 :green_heart: compile 0m 33s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: javac 0m 33s hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 41 unchanged - 2 fixed = 41 total (was 43)
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 24s the patch passed
+1 :green_heart: mvnsite 0m 38s the patch passed
+1 :green_heart: javadoc 0m 19s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 28s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 11s the patch passed
+1 :green_heart: shadedclient 23m 37s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 36s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 42s The patch does not generate ASF License warnings.
104m 17s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/8/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/4698
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname Linux 72276699e805 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / fa3775c0fe49b77750d3ee3e88c313331be4a596
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/8/testReport/
Max. process+thread count 600 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/8/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 11 '22 12:08 hadoop-yetus

Thanks @steveloughran, I've suppressed deprecation warnings and used LogExactlyOnce. Yetus is happy now so this is now ready for review. Also had a couple of questions.

ahmarsuhail avatar Aug 11 '22 12:08 ahmarsuhail

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 1m 2s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 1s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+0 :ok: markdownlint 0m 1s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 13 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 38m 1s trunk passed
+1 :green_heart: compile 1m 1s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: compile 0m 55s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 0m 53s trunk passed
+1 :green_heart: mvnsite 1m 3s trunk passed
+1 :green_heart: javadoc 0m 49s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 50s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 35s trunk passed
+1 :green_heart: shadedclient 21m 5s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 40s the patch passed
+1 :green_heart: compile 0m 44s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javac 0m 44s hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 42 unchanged - 2 fixed = 42 total (was 44)
+1 :green_heart: compile 0m 36s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: javac 0m 36s hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 41 unchanged - 2 fixed = 41 total (was 43)
-1 :x: blanks 0m 0s /blanks-eol.txt The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 :green_heart: checkstyle 0m 28s the patch passed
+1 :green_heart: mvnsite 0m 41s the patch passed
+1 :green_heart: javadoc 0m 24s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 32s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 13s the patch passed
+1 :green_heart: shadedclient 20m 26s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 3m 3s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 50s The patch does not generate ASF License warnings.
98m 7s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/9/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/4698
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname Linux bfb637ac36d2 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / fe5e2e3ea361ff8bae153d0bcb68fd8890e0e3ed
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/9/testReport/
Max. process+thread count 729 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/9/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 11 '22 13:08 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 47s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 1s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+0 :ok: markdownlint 0m 0s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 13 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 41m 42s trunk passed
+1 :green_heart: compile 0m 53s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: compile 0m 45s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 0m 43s trunk passed
+1 :green_heart: mvnsite 0m 52s trunk passed
+1 :green_heart: javadoc 0m 38s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 40s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 30s trunk passed
+1 :green_heart: shadedclient 23m 55s branch has no errors when building and testing our client artifacts.
-0 :warning: patch 24m 17s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 35s the patch passed
+1 :green_heart: compile 0m 41s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javac 0m 41s hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 42 unchanged - 2 fixed = 42 total (was 44)
+1 :green_heart: compile 0m 32s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: javac 0m 32s hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 41 unchanged - 2 fixed = 41 total (was 43)
-1 :x: blanks 0m 0s /blanks-eol.txt The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 :green_heart: checkstyle 0m 23s the patch passed
+1 :green_heart: mvnsite 0m 38s the patch passed
+1 :green_heart: javadoc 0m 19s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 28s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 13s the patch passed
+1 :green_heart: shadedclient 23m 19s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 41s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 42s The patch does not generate ASF License warnings.
105m 3s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/10/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/4698
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname Linux 9028e0c9900e 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 4c652b1d88b6ee21e76eaef30f752836377d5354
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/10/testReport/
Max. process+thread count 530 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/10/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 18 '22 15:08 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 45s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 1s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+0 :ok: markdownlint 0m 0s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 13 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 39m 4s trunk passed
+1 :green_heart: compile 1m 0s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: compile 0m 53s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 0m 53s trunk passed
+1 :green_heart: mvnsite 1m 2s trunk passed
+1 :green_heart: javadoc 0m 48s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 50s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 34s trunk passed
+1 :green_heart: shadedclient 20m 57s branch has no errors when building and testing our client artifacts.
-0 :warning: patch 21m 24s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 40s the patch passed
+1 :green_heart: compile 0m 43s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
-1 :x: javac 0m 43s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 15 new + 42 unchanged - 2 fixed = 57 total (was 44)
+1 :green_heart: compile 0m 37s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 :x: javac 0m 37s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 15 new + 41 unchanged - 2 fixed = 56 total (was 43)
-1 :x: blanks 0m 1s /blanks-eol.txt The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 :green_heart: checkstyle 0m 28s the patch passed
+1 :green_heart: mvnsite 0m 41s the patch passed
+1 :green_heart: javadoc 0m 23s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 31s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 14s the patch passed
+1 :green_heart: shadedclient 20m 26s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 49s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 51s The patch does not generate ASF License warnings.
98m 41s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/11/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/4698
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname Linux 696513c54a6f 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / b385452ae8924521282a93af6160c19fcbf19469
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/11/testReport/
Max. process+thread count 727 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/11/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 18 '22 16:08 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 1m 6s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 1s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+0 :ok: markdownlint 0m 0s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 13 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 40m 22s trunk passed
+1 :green_heart: compile 0m 56s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: compile 0m 50s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 0m 43s trunk passed
+1 :green_heart: mvnsite 0m 53s trunk passed
+1 :green_heart: javadoc 0m 40s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 41s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 30s trunk passed
+1 :green_heart: shadedclient 21m 6s branch has no errors when building and testing our client artifacts.
-0 :warning: patch 21m 33s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 39s the patch passed
+1 :green_heart: compile 0m 43s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
-1 :x: javac 0m 43s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 15 new + 42 unchanged - 2 fixed = 57 total (was 44)
+1 :green_heart: compile 0m 37s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 :x: javac 0m 37s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 15 new + 41 unchanged - 2 fixed = 56 total (was 43)
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 0m 28s /results-checkstyle-hadoop-tools_hadoop-aws.txt hadoop-tools/hadoop-aws: The patch generated 1 new + 28 unchanged - 0 fixed = 29 total (was 28)
+1 :green_heart: mvnsite 0m 42s the patch passed
+1 :green_heart: javadoc 0m 24s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 32s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 14s the patch passed
+1 :green_heart: shadedclient 20m 44s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 3m 6s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 49s The patch does not generate ASF License warnings.
100m 23s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/12/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/4698
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname Linux b3ba1b2b9461 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 8ded46fafe1044db9b4cde71765b89a07beb5a46
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/12/testReport/
Max. process+thread count 558 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/12/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 18 '22 17:08 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 49s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 1s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+0 :ok: markdownlint 0m 0s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 13 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 42m 14s trunk passed
+1 :green_heart: compile 0m 51s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: compile 0m 46s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 0m 43s trunk passed
+1 :green_heart: mvnsite 0m 52s trunk passed
+1 :green_heart: javadoc 0m 38s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 40s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 28s trunk passed
+1 :green_heart: shadedclient 23m 52s branch has no errors when building and testing our client artifacts.
-0 :warning: patch 24m 14s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 35s the patch passed
+1 :green_heart: compile 0m 42s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
-1 :x: javac 0m 42s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 15 new + 42 unchanged - 2 fixed = 57 total (was 44)
+1 :green_heart: compile 0m 33s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 :x: javac 0m 33s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 15 new + 41 unchanged - 2 fixed = 56 total (was 43)
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 0m 23s /results-checkstyle-hadoop-tools_hadoop-aws.txt hadoop-tools/hadoop-aws: The patch generated 1 new + 28 unchanged - 0 fixed = 29 total (was 28)
+1 :green_heart: mvnsite 0m 38s the patch passed
+1 :green_heart: javadoc 0m 20s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 28s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 12s the patch passed
+1 :green_heart: shadedclient 23m 42s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 43s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 42s The patch does not generate ASF License warnings.
106m 4s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/13/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/4698
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname Linux 70f84b1803e2 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 8ded46fafe1044db9b4cde71765b89a07beb5a46
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/13/testReport/
Max. process+thread count 528 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/13/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 18 '22 17:08 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 56s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 1s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+0 :ok: markdownlint 0m 1s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 13 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 38m 51s trunk passed
+1 :green_heart: compile 1m 1s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: compile 0m 54s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 0m 53s trunk passed
+1 :green_heart: mvnsite 1m 1s trunk passed
+1 :green_heart: javadoc 0m 48s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 50s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 35s trunk passed
+1 :green_heart: shadedclient 21m 6s branch has no errors when building and testing our client artifacts.
-0 :warning: patch 21m 33s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 39s the patch passed
+1 :green_heart: compile 0m 43s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
-1 :x: javac 0m 43s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 15 new + 42 unchanged - 2 fixed = 57 total (was 44)
+1 :green_heart: compile 0m 38s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 :x: javac 0m 37s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 15 new + 41 unchanged - 2 fixed = 56 total (was 43)
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 26s the patch passed
+1 :green_heart: mvnsite 0m 43s the patch passed
+1 :green_heart: javadoc 0m 22s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 31s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 13s the patch passed
+1 :green_heart: shadedclient 20m 34s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 3m 3s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 51s The patch does not generate ASF License warnings.
98m 50s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/14/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/4698
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname Linux 2d61168a879a 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 71ac6f8541df4be58960ef0b213a0e6db0a81b60
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/14/testReport/
Max. process+thread count 560 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/14/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 19 '22 10:08 hadoop-yetus

@steveloughran have added a new V2Migration class and reverted using fully qualified classnames. Some deprecation warnings now, but they are all for imports.

For AWSCredentialProviderList, this is what I was thinking too. We can definitely prevent DT from breaking..but since usage is limited not sure if it's better to force people to update instead. have marked the class final for now.

there's also a patch warning on yetus now..not sure why? i'd merged trunk into this branch a couple of days ago.

ahmarsuhail avatar Aug 19 '22 12:08 ahmarsuhail

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 53s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 1s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+0 :ok: markdownlint 0m 0s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 13 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 43m 31s trunk passed
+1 :green_heart: compile 1m 28s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: compile 1m 9s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 1m 5s trunk passed
+1 :green_heart: mvnsite 1m 13s trunk passed
+1 :green_heart: javadoc 0m 45s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 52s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 2m 19s trunk passed
+1 :green_heart: shadedclient 26m 15s branch has no errors when building and testing our client artifacts.
-0 :warning: patch 26m 37s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 35s the patch passed
+1 :green_heart: compile 0m 41s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
-1 :x: javac 0m 41s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 15 new + 42 unchanged - 2 fixed = 57 total (was 44)
+1 :green_heart: compile 0m 34s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 :x: javac 0m 33s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 15 new + 41 unchanged - 2 fixed = 56 total (was 43)
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 23s the patch passed
+1 :green_heart: mvnsite 0m 38s the patch passed
+1 :green_heart: javadoc 0m 19s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 27s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 12s the patch passed
+1 :green_heart: shadedclient 23m 28s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 47s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 42s The patch does not generate ASF License warnings.
111m 53s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/15/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/4698
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname Linux 4bae8a53d06c 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / bd80a4f74e4457a075af1014b8302297f809cbd1
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/15/testReport/
Max. process+thread count 529 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/15/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 22 '22 12:08 hadoop-yetus

Looks good. Be aware that in CDP we will not backport this without the logs disabled, unless/until we have our own code all happy, which will involve a move to the v2 sdk elsewhere (STS interaction).

getClient()

the s3a FS class has a getClient() call too, which is only used internally or in external code which is doing things to get at it (the forTesting() one came in later, see).

The internal stuff could make that private and instead pass it in to WriteOperationsHelper.

really that class should

  1. have the Select operation moved to its own interface and used exclusively by SelectBinding
  2. have an interface of all the ops it is allowed to call on the s3a fs.

None of our subsidiary classes should have direct refs to the owning fs; it just complicates our life too much. seemed a good idea at the time though...

what does that mean for that patch? nothing, except by moving the (split) callbacks into two interfaces/impls. that getClient call could be made private and migration would be easier.

datatypes

Does the migration also mean that all the datatypes, s3Object, multipart input stream etc are all going to go too?

steveloughran avatar Aug 22 '22 16:08 steveloughran

Thanks @steveloughran, have made the suggested changes.

And yes, data types are changing. getObject now returns ResponseInputStream<GetObjectResponse>.

In this PR, I'm logging a warning on getObjectMetadata, which will now return HeadObjectResponse. Javadocs say this method is used by external applications. Are there any other such methods in S3AFS that may return AWS data types directly?

ahmarsuhail avatar Aug 24 '22 13:08 ahmarsuhail

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 52s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 1s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+0 :ok: markdownlint 0m 1s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 16 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 41m 43s trunk passed
+1 :green_heart: compile 0m 52s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: compile 0m 42s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 0m 39s trunk passed
+1 :green_heart: mvnsite 0m 49s trunk passed
+1 :green_heart: javadoc 0m 36s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 37s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 23s trunk passed
-1 :x: shadedclient 2m 29s branch has errors when building and testing our client artifacts.
-0 :warning: patch 2m 50s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 34s the patch passed
+1 :green_heart: compile 0m 39s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
-1 :x: javac 0m 39s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 8 new + 42 unchanged - 2 fixed = 50 total (was 44)
+1 :green_heart: compile 0m 31s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 :x: javac 0m 31s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 8 new + 41 unchanged - 2 fixed = 49 total (was 43)
-1 :x: blanks 0m 0s /blanks-eol.txt The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 :green_heart: checkstyle 0m 22s the patch passed
+1 :green_heart: mvnsite 0m 37s the patch passed
+1 :green_heart: javadoc 0m 17s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
-1 :x: javadoc 0m 26s /results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 1 new + 37 unchanged - 0 fixed = 38 total (was 37)
+1 :green_heart: spotbugs 1m 11s the patch passed
-1 :x: shadedclient 1m 56s patch has errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 34s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 33s The patch does not generate ASF License warnings.
61m 36s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/16/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/4698
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname Linux 3257ec3f8d98 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / eb0666a406ba53901552c0ae43c93d5875fdc08d
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/16/testReport/
Max. process+thread count 257 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/16/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 24 '22 14:08 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 52s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+0 :ok: markdownlint 0m 1s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 16 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 41m 32s trunk passed
+1 :green_heart: compile 0m 49s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: compile 0m 41s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: checkstyle 0m 39s trunk passed
+1 :green_heart: mvnsite 0m 51s trunk passed
+1 :green_heart: javadoc 0m 36s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 37s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 25s trunk passed
-1 :x: shadedclient 2m 32s branch has errors when building and testing our client artifacts.
-0 :warning: patch 2m 53s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 34s the patch passed
+1 :green_heart: compile 0m 38s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
-1 :x: javac 0m 38s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 8 new + 42 unchanged - 2 fixed = 50 total (was 44)
+1 :green_heart: compile 0m 32s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 :x: javac 0m 32s /results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 8 new + 41 unchanged - 2 fixed = 49 total (was 43)
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 21s the patch passed
+1 :green_heart: mvnsite 0m 36s the patch passed
+1 :green_heart: javadoc 0m 17s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 :green_heart: javadoc 0m 26s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 :green_heart: spotbugs 1m 10s the patch passed
-1 :x: shadedclient 1m 56s patch has errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 35s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 33s The patch does not generate ASF License warnings.
61m 21s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/17/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/4698
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname Linux 067ff499a90a 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 3b87159add00a8820e87942cae7462a7fc354390
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/17/testReport/
Max. process+thread count 258 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/17/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 24 '22 16:08 hadoop-yetus

@steveloughran i think you may have viewed changes from a previous commit, may last commit addresses both of those issues: https://github.com/apache/hadoop/pull/4698/commits/3b87159add00a8820e87942cae7462a7fc354390

ahmarsuhail avatar Aug 25 '22 11:08 ahmarsuhail

oh, you are right. ready to go in...yetus is just complaining about deprecation.

+1, merging

steveloughran avatar Aug 25 '22 15:08 steveloughran