hadoop icon indicating copy to clipboard operation
hadoop copied to clipboard

[HADOOP-18562]: S3A: support custom S3 and STS headers.

Open PradhanPrerak opened this issue 1 year ago • 14 comments

Description of PR

  • Support to add custom headers to s3a s3 and sts client

Introducing new custom.headers config for each client type, Upon setting this header each s3/sts request will set these custom headers by setting overrideClientConfig on clients

"fs.s3a.client.sts.custom.headers" = "Header1:Value1:Value2,Header2:Value1"
"fs.s3a.client.s3.custom.headers" = "Header1:Value1:Value2,Header2:Value1"

How was this patch tested?

Ran integ tests using mvn -Dparallel-tests -DtestsThreadCount=8 clean verify for a bucket in us-west-2 UTs added which succeeded as well

For code changes:

  • [ Y ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • [ Y ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • [ N/A ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • [ N/A ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

PradhanPrerak avatar Feb 14 '24 04:02 PradhanPrerak

Can use HADOOP-18562 for the JIRA ID here; hadoop codebase see. thanks

steveloughran avatar Mar 19 '24 17:03 steveloughran

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 1m 4s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 1s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 52m 48s trunk passed
+1 :green_heart: compile 0m 45s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: compile 0m 35s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: checkstyle 0m 31s trunk passed
+1 :green_heart: mvnsite 0m 44s trunk passed
+1 :green_heart: javadoc 0m 27s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 0m 33s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 1m 11s trunk passed
+1 :green_heart: shadedclient 40m 35s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 32s the patch passed
+1 :green_heart: compile 0m 41s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javac 0m 41s the patch passed
+1 :green_heart: compile 0m 27s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: javac 0m 27s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 0m 20s /results-checkstyle-hadoop-tools_hadoop-aws.txt hadoop-tools/hadoop-aws: The patch generated 8 new + 2 unchanged - 0 fixed = 10 total (was 2)
+1 :green_heart: mvnsite 0m 32s the patch passed
+1 :green_heart: javadoc 0m 15s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 0m 26s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 1m 11s the patch passed
+1 :green_heart: shadedclient 39m 38s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 3m 5s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 38s The patch does not generate ASF License warnings.
148m 25s
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/2/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6550
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux e176d7f14c3c 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / cfe1f7ccda00ff38061a539eb4a677d279aa73dc
Default Java Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/2/testReport/
Max. process+thread count 527 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/2/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Jun 13 '24 00:06 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 17m 22s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
-1 :x: test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 50m 25s trunk passed
+1 :green_heart: compile 0m 44s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: compile 0m 34s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: checkstyle 0m 31s trunk passed
+1 :green_heart: mvnsite 0m 42s trunk passed
+1 :green_heart: javadoc 0m 27s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 0m 35s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 1m 9s trunk passed
+1 :green_heart: shadedclient 39m 10s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 29s the patch passed
+1 :green_heart: compile 0m 35s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javac 0m 35s the patch passed
+1 :green_heart: compile 0m 27s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: javac 0m 27s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 0m 21s /results-checkstyle-hadoop-tools_hadoop-aws.txt hadoop-tools/hadoop-aws: The patch generated 8 new + 2 unchanged - 0 fixed = 10 total (was 2)
+1 :green_heart: mvnsite 0m 32s the patch passed
+1 :green_heart: javadoc 0m 16s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 0m 25s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 1m 8s the patch passed
+1 :green_heart: shadedclient 39m 25s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 52s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 37s The patch does not generate ASF License warnings.
160m 7s
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/3/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6550
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 9edefc1f85b0 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 7f58d07ef5540516a61c1fa14566247c9e2b3925
Default Java Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/3/testReport/
Max. process+thread count 527 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/3/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Jun 17 '24 12:06 hadoop-yetus

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 46s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 1s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 49m 47s trunk passed
+1 :green_heart: compile 0m 44s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: compile 0m 35s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: checkstyle 0m 32s trunk passed
+1 :green_heart: mvnsite 0m 41s trunk passed
+1 :green_heart: javadoc 0m 27s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 0m 34s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 1m 10s trunk passed
+1 :green_heart: shadedclient 39m 5s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 29s the patch passed
+1 :green_heart: compile 0m 36s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javac 0m 36s the patch passed
+1 :green_heart: compile 0m 26s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: javac 0m 26s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 0m 21s /results-checkstyle-hadoop-tools_hadoop-aws.txt hadoop-tools/hadoop-aws: The patch generated 15 new + 3 unchanged - 0 fixed = 18 total (was 3)
+1 :green_heart: mvnsite 0m 33s the patch passed
+1 :green_heart: javadoc 0m 16s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 0m 25s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 1m 9s the patch passed
+1 :green_heart: shadedclient 38m 59s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 51s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 38s The patch does not generate ASF License warnings.
142m 20s
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/4/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6550
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux d6e92357719b 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 917e21417af5979b42a9887862efa56db493c16d
Default Java Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/4/testReport/
Max. process+thread count 528 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/4/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Jun 17 '24 23:06 hadoop-yetus

thanks. we didn't kneed to know all the secrets, just aws "us-west-2". something like

tested us-west-2 standard bucket with -Dparallel-tests -DtestsThreadCount=10. if its got versioning enabled, accessed via fips endoint etc, that's also nice to know as it shows we are getting more test coverage.

steveloughran avatar Jun 25 '24 19:06 steveloughran

thanks. we didn't kneed to know all the secrets, just aws "us-west-2". something like

tested us-west-2 standard bucket with -Dparallel-tests -DtestsThreadCount=10. if its got versioning enabled, accessed via fips endoint etc, that's also nice to know as it shows we are getting more test coverage.

got it, will keep in mind

PradhanPrerak avatar Jul 02 '24 19:07 PradhanPrerak

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 47s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 50m 18s trunk passed
+1 :green_heart: compile 0m 43s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: compile 0m 34s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: checkstyle 0m 32s trunk passed
+1 :green_heart: mvnsite 0m 42s trunk passed
+1 :green_heart: javadoc 0m 28s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 0m 33s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 1m 8s trunk passed
+1 :green_heart: shadedclient 39m 16s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 29s the patch passed
+1 :green_heart: compile 0m 36s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javac 0m 35s the patch passed
+1 :green_heart: compile 0m 26s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: javac 0m 26s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 0m 21s /results-checkstyle-hadoop-tools_hadoop-aws.txt hadoop-tools/hadoop-aws: The patch generated 15 new + 3 unchanged - 0 fixed = 18 total (was 3)
+1 :green_heart: mvnsite 0m 31s the patch passed
+1 :green_heart: javadoc 0m 15s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 0m 25s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 1m 8s the patch passed
+1 :green_heart: shadedclient 39m 41s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 33s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 36s The patch does not generate ASF License warnings.
143m 26s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/5/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6550
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux f23fe868c010 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / e8ed19aa82bc0006ea9e23c1a0349fa0a61c2992
Default Java Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/5/testReport/
Max. process+thread count 534 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/5/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Jul 02 '24 21:07 hadoop-yetus

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 46s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 51m 10s trunk passed
+1 :green_heart: compile 0m 43s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: compile 0m 35s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: checkstyle 0m 32s trunk passed
+1 :green_heart: mvnsite 0m 41s trunk passed
+1 :green_heart: javadoc 0m 27s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 0m 34s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 1m 8s trunk passed
+1 :green_heart: shadedclient 39m 24s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 42s the patch passed
+1 :green_heart: compile 0m 36s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javac 0m 36s the patch passed
+1 :green_heart: compile 0m 27s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: javac 0m 27s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 0m 22s /results-checkstyle-hadoop-tools_hadoop-aws.txt hadoop-tools/hadoop-aws: The patch generated 16 new + 3 unchanged - 0 fixed = 19 total (was 3)
+1 :green_heart: mvnsite 0m 31s the patch passed
+1 :green_heart: javadoc 0m 16s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 0m 25s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 1m 9s the patch passed
+1 :green_heart: shadedclient 38m 53s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 33s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 36s The patch does not generate ASF License warnings.
143m 52s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/6/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6550
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 466c384a020a 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 626d8b02b0753f22cd6f0c8741a2b1f1b25438a9
Default Java Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/6/testReport/
Max. process+thread count 556 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/6/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Jul 03 '24 01:07 hadoop-yetus

@PradhanPrerak - Please fix the checkstyle issues as mentioned here (https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/6/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)

shameersss1 avatar Jul 03 '24 05:07 shameersss1

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 48s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 49m 59s trunk passed
+1 :green_heart: compile 0m 44s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: compile 0m 35s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: checkstyle 0m 32s trunk passed
+1 :green_heart: mvnsite 0m 43s trunk passed
+1 :green_heart: javadoc 0m 27s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 0m 35s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 1m 11s trunk passed
+1 :green_heart: shadedclient 39m 11s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 30s the patch passed
+1 :green_heart: compile 0m 35s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javac 0m 35s the patch passed
+1 :green_heart: compile 0m 26s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: javac 0m 26s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 20s the patch passed
+1 :green_heart: mvnsite 0m 32s the patch passed
+1 :green_heart: javadoc 0m 15s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 0m 25s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 1m 8s the patch passed
+1 :green_heart: shadedclient 39m 2s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 31s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 38s The patch does not generate ASF License warnings.
142m 27s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/9/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6550
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux e1fe23fbea64 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 0fba78f53e9745f8436d460a4ece86ddba9ceae8
Default Java Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/9/testReport/
Max. process+thread count 570 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/9/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Jul 03 '24 23:07 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 6m 57s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
-1 :x: mvninstall 32m 41s /branch-mvninstall-root.txt root in trunk failed.
+1 :green_heart: compile 0m 27s trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: compile 0m 22s trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: checkstyle 0m 22s trunk passed
+1 :green_heart: mvnsite 0m 30s trunk passed
+1 :green_heart: javadoc 0m 20s trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: javadoc 0m 23s trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: spotbugs 0m 48s trunk passed
+1 :green_heart: shadedclient 20m 30s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 18s the patch passed
+1 :green_heart: compile 0m 21s the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: javac 0m 21s the patch passed
+1 :green_heart: compile 0m 15s the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: javac 0m 15s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 13s the patch passed
+1 :green_heart: mvnsite 0m 23s the patch passed
+1 :green_heart: javadoc 0m 28s the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: javadoc 0m 19s the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: spotbugs 0m 45s the patch passed
+1 :green_heart: shadedclient 20m 22s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 1m 58s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 25s The patch does not generate ASF License warnings.
89m 52s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/10/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6550
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 394bc0fbd6b0 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / d364ac44998098ba887df5d906c97e230f85df95
Default Java Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/10/testReport/
Max. process+thread count 551 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/10/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 15 '24 00:08 hadoop-yetus

yetus failure is in Apache Hadoop YARN Application Catalog Webapp seems unrelated

PradhanPrerak avatar Aug 15 '24 00:08 PradhanPrerak

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 17m 53s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 :green_heart: mvninstall 51m 6s trunk passed
+1 :green_heart: compile 0m 44s trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: compile 0m 34s trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: checkstyle 0m 32s trunk passed
+1 :green_heart: mvnsite 0m 44s trunk passed
+1 :green_heart: javadoc 0m 28s trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: javadoc 0m 33s trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: spotbugs 1m 10s trunk passed
+1 :green_heart: shadedclient 39m 11s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 0m 30s the patch passed
+1 :green_heart: compile 0m 36s the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: javac 0m 36s the patch passed
+1 :green_heart: compile 0m 27s the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: javac 0m 27s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 20s the patch passed
+1 :green_heart: mvnsite 0m 31s the patch passed
+1 :green_heart: javadoc 0m 16s the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: javadoc 0m 25s the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: spotbugs 1m 8s the patch passed
+1 :green_heart: shadedclient 39m 22s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 38s hadoop-aws in the patch passed.
+1 :green_heart: asflicense 0m 37s The patch does not generate ASF License warnings.
161m 0s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/11/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6550
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 1790bccc070f 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 67416252985ba907e8fdcf429d45d8daf0453aba
Default Java Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/11/testReport/
Max. process+thread count 527 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6550/11/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 15 '24 02:08 hadoop-yetus