hadoop icon indicating copy to clipboard operation
hadoop copied to clipboard

HADOOP-19131. Assist reflection IO with WrappedOperations class

Open steveloughran opened this issue 1 year ago • 9 comments

HADOOP-19131 Assist reflection IO with WrappedOperations class

How was this patch tested?

Needs new tests going through reflection, maybe some in openfile contract to guarantee full use.

For code changes:

  • [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • [ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

steveloughran avatar Mar 28 '24 16:03 steveloughran

prepared parquet for this by renaming vectorio package to org.apache.parquet.hadoop.util.wrappedio

steveloughran avatar Apr 02 '24 18:04 steveloughran

@mukund-thakur this pr renames bulkDelete_PageSize to bulkDelete_pageSize to be consistent with everything else.

My iceberg PR https://github.com/apache/iceberg/pull/10233 looks for the new name; it is now dynamic and should build link up if we can think of a way to test it (proposed: make it an option to use if present), default is true.

steveloughran avatar May 29 '24 17:05 steveloughran

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 33s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+0 :ok: xmllint 0m 0s xmllint was not available.
+0 :ok: markdownlint 0m 0s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 10 new or modified test files.
_ trunk Compile Tests _
+0 :ok: mvndep 14m 48s Maven dependency ordering for branch
+1 :green_heart: mvninstall 32m 40s trunk passed
+1 :green_heart: compile 19m 14s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: compile 17m 59s trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: checkstyle 4m 30s trunk passed
+1 :green_heart: mvnsite 5m 4s trunk passed
+1 :green_heart: javadoc 3m 46s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 4m 15s trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: spotbugs 8m 58s trunk passed
+1 :green_heart: shadedclient 34m 59s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 34s Maven dependency ordering for patch
+1 :green_heart: mvninstall 3m 22s the patch passed
+1 :green_heart: compile 18m 35s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javac 18m 35s the patch passed
+1 :green_heart: compile 17m 49s the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: javac 17m 49s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 4m 26s /results-checkstyle-root.txt root: The patch generated 83 new + 40 unchanged - 3 fixed = 123 total (was 43)
+1 :green_heart: mvnsite 5m 2s the patch passed
-1 :x: javadoc 1m 7s /patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt hadoop-common in the patch failed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.
+1 :green_heart: javadoc 4m 18s the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: spotbugs 9m 37s the patch passed
+1 :green_heart: shadedclient 34m 56s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 20m 19s hadoop-common in the patch passed.
-1 :x: unit 234m 18s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 :green_heart: unit 3m 29s hadoop-aws in the patch passed.
+1 :green_heart: unit 2m 50s hadoop-azure in the patch passed.
-1 :x: asflicense 1m 14s /results-asflicense.txt The patch generated 3 ASF License warnings.
515m 51s
Reason Tests
Failed junit tests hadoop.hdfs.server.namenode.TestCacheDirectives
hadoop.hdfs.server.namenode.TestRefreshNamenodeReplicationConfig
hadoop.hdfs.server.namenode.TestReconstructStripedBlocks
hadoop.fs.contract.hdfs.TestDFSWrappedIO
hadoop.hdfs.server.namenode.TestFsck
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/23/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6686
Optional Tests dupname asflicense codespell detsecrets xmllint compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle markdownlint
uname Linux d045111a54be 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 82b4e703913ae0a2fa9dec2a75c51398fda59393
Default Java Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/23/testReport/
Max. process+thread count 3602 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/23/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Jun 14 '24 01:06 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 32s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+0 :ok: xmllint 0m 0s xmllint was not available.
+0 :ok: markdownlint 0m 1s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 10 new or modified test files.
_ trunk Compile Tests _
+0 :ok: mvndep 14m 37s Maven dependency ordering for branch
+1 :green_heart: mvninstall 32m 47s trunk passed
+1 :green_heart: compile 19m 6s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: compile 17m 48s trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: checkstyle 4m 28s trunk passed
+1 :green_heart: mvnsite 5m 3s trunk passed
+1 :green_heart: javadoc 3m 41s trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javadoc 4m 12s trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: spotbugs 8m 45s trunk passed
+1 :green_heart: shadedclient 34m 35s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 32s Maven dependency ordering for patch
+1 :green_heart: mvninstall 3m 20s the patch passed
+1 :green_heart: compile 18m 46s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 :green_heart: javac 18m 46s the patch passed
+1 :green_heart: compile 17m 46s the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: javac 17m 46s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 4m 39s /results-checkstyle-root.txt root: The patch generated 83 new + 40 unchanged - 3 fixed = 123 total (was 43)
+1 :green_heart: mvnsite 5m 5s the patch passed
-1 :x: javadoc 1m 6s /patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt hadoop-common in the patch failed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.
+1 :green_heart: javadoc 4m 17s the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 :green_heart: spotbugs 9m 45s the patch passed
+1 :green_heart: shadedclient 34m 53s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 20m 10s hadoop-common in the patch passed.
-1 :x: unit 239m 33s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 :green_heart: unit 3m 29s hadoop-aws in the patch passed.
+1 :green_heart: unit 2m 49s hadoop-azure in the patch passed.
-1 :x: asflicense 1m 15s /results-asflicense.txt The patch generated 1 ASF License warnings.
520m 15s
Reason Tests
Failed junit tests hadoop.hdfs.server.namenode.TestCacheDirectives
hadoop.hdfs.server.namenode.TestReconstructStripedBlocks
hadoop.fs.contract.hdfs.TestDFSWrappedIO
hadoop.hdfs.server.namenode.TestNamenodeRetryCache
hadoop.hdfs.server.namenode.TestFsck
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/24/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6686
Optional Tests dupname asflicense codespell detsecrets xmllint compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle markdownlint
uname Linux c177fa3b72b7 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 82b4e703913ae0a2fa9dec2a75c51398fda59393
Default Java Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/24/testReport/
Max. process+thread count 4035 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/24/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Jun 14 '24 01:06 hadoop-yetus

see also #6885

steveloughran avatar Jun 14 '24 10:06 steveloughran

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 14m 59s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+0 :ok: xmllint 0m 0s xmllint was not available.
+0 :ok: markdownlint 0m 1s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 10 new or modified test files.
_ trunk Compile Tests _
+0 :ok: mvndep 14m 50s Maven dependency ordering for branch
+1 :green_heart: mvninstall 32m 11s trunk passed
+1 :green_heart: compile 17m 22s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: compile 16m 6s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: checkstyle 4m 31s trunk passed
+1 :green_heart: mvnsite 5m 20s trunk passed
+1 :green_heart: javadoc 3m 59s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 4m 34s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 8m 33s trunk passed
+1 :green_heart: shadedclient 34m 23s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 34s Maven dependency ordering for patch
+1 :green_heart: mvninstall 3m 10s the patch passed
+1 :green_heart: compile 16m 43s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javac 16m 43s the patch passed
+1 :green_heart: compile 15m 57s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: javac 15m 57s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 4m 29s /results-checkstyle-root.txt root: The patch generated 55 new + 40 unchanged - 3 fixed = 95 total (was 43)
+1 :green_heart: mvnsite 5m 14s the patch passed
+1 :green_heart: javadoc 3m 58s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 4m 30s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 9m 25s the patch passed
+1 :green_heart: shadedclient 34m 33s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 19m 49s hadoop-common in the patch passed.
-1 :x: unit 234m 31s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
-1 :x: unit 1m 18s /patch-unit-hadoop-tools_hadoop-aws.txt hadoop-aws in the patch passed.
-1 :x: unit 0m 55s /patch-unit-hadoop-tools_hadoop-azure.txt hadoop-azure in the patch failed.
+0 :ok: asflicense 0m 48s ASF License check generated no output?
517m 40s
Reason Tests
Failed junit tests hadoop.hdfs.server.namenode.TestFSImageWithXAttr
hadoop.hdfs.server.namenode.TestCacheDirectivesWithViewDFS
hadoop.hdfs.server.namenode.TestReencryption
hadoop.hdfs.server.namenode.TestReencryptionWithKMS
hadoop.hdfs.server.namenode.TestQuotaWithStripedBlocksWithRandomECPolicy
hadoop.hdfs.server.namenode.TestCheckpoint
hadoop.hdfs.server.namenode.TestAuditLogs
hadoop.hdfs.server.namenode.TestFileTruncate
hadoop.hdfs.server.namenode.TestBlockPlacementPolicyRackFaultTolerant
hadoop.hdfs.server.namenode.TestCacheDirectives
hadoop.hdfs.server.namenode.TestAuditLoggerWithCommands
hadoop.hdfs.server.namenode.TestParallelImageWrite
hadoop.fs.s3a.audit.impl.TestActiveAuditManagerThreadLeakage
hadoop.fs.s3a.TestS3AccessGrantConfiguration
Subsystem Report/Notes
Docker ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/25/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6686
Optional Tests dupname asflicense codespell detsecrets xmllint compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle markdownlint
uname Linux 3fbafeaa102c 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 0f436d5f0ec3783cf1c52777661930b0b03e1732
Default Java Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/25/testReport/
Max. process+thread count 3513 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/25/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Jun 21 '24 00:06 hadoop-yetus

thread memory problems triggering test failures

steveloughran avatar Jun 21 '24 15:06 steveloughran

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 32s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+0 :ok: xmllint 0m 0s xmllint was not available.
+0 :ok: markdownlint 0m 1s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 13 new or modified test files.
_ trunk Compile Tests _
+0 :ok: mvndep 14m 35s Maven dependency ordering for branch
+1 :green_heart: mvninstall 32m 14s trunk passed
+1 :green_heart: compile 17m 34s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: compile 16m 14s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: checkstyle 4m 26s trunk passed
+1 :green_heart: mvnsite 6m 2s trunk passed
+1 :green_heart: javadoc 4m 46s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 5m 6s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 9m 20s trunk passed
+1 :green_heart: shadedclient 35m 32s branch has no errors when building and testing our client artifacts.
-0 :warning: patch 35m 58s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 33s Maven dependency ordering for patch
+1 :green_heart: mvninstall 3m 37s the patch passed
+1 :green_heart: compile 16m 45s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javac 16m 45s the patch passed
+1 :green_heart: compile 16m 20s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: javac 16m 20s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 4m 24s /results-checkstyle-root.txt root: The patch generated 60 new + 66 unchanged - 3 fixed = 126 total (was 69)
+1 :green_heart: mvnsite 5m 55s the patch passed
+1 :green_heart: javadoc 4m 43s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 5m 17s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 10m 34s the patch passed
+1 :green_heart: shadedclient 34m 22s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 19m 45s hadoop-common in the patch passed.
-1 :x: unit 224m 47s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 :green_heart: unit 3m 2s hadoop-aws in the patch passed.
+1 :green_heart: unit 2m 54s hadoop-azure in the patch passed.
+1 :green_heart: unit 0m 51s hadoop-aliyun in the patch passed.
+1 :green_heart: asflicense 1m 16s The patch does not generate ASF License warnings.
507m 0s
Reason Tests
Failed junit tests hadoop.hdfs.server.namenode.TestFSNamesystemLockReport
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/30/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6686
Optional Tests dupname asflicense codespell detsecrets xmllint compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle markdownlint
uname Linux 9368a7824fcf 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / b44df10afcf5c9cfc8d542a027f2d42c9bca44a8
Default Java Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/30/testReport/
Max. process+thread count 3432 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure hadoop-tools/hadoop-aliyun U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/30/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Jul 16 '24 23:07 hadoop-yetus

test failure is org.apache.hadoop.hdfs.server.namenode.TestFSNamesystemLockReport.test

steveloughran avatar Jul 17 '24 19:07 steveloughran

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 31s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 1s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+0 :ok: xmllint 0m 1s xmllint was not available.
+0 :ok: markdownlint 0m 0s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 13 new or modified test files.
_ trunk Compile Tests _
+0 :ok: mvndep 15m 27s Maven dependency ordering for branch
+1 :green_heart: mvninstall 33m 19s trunk passed
+1 :green_heart: compile 17m 27s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: compile 16m 20s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: checkstyle 4m 26s trunk passed
+1 :green_heart: mvnsite 6m 0s trunk passed
+1 :green_heart: javadoc 4m 52s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 5m 16s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 9m 28s trunk passed
+1 :green_heart: shadedclient 34m 6s branch has no errors when building and testing our client artifacts.
-0 :warning: patch 34m 33s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 33s Maven dependency ordering for patch
+1 :green_heart: mvninstall 3m 35s the patch passed
+1 :green_heart: compile 16m 53s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javac 16m 53s the patch passed
+1 :green_heart: compile 16m 15s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: javac 16m 15s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 4m 27s /results-checkstyle-root.txt root: The patch generated 59 new + 66 unchanged - 3 fixed = 125 total (was 69)
+1 :green_heart: mvnsite 5m 59s the patch passed
-1 :x: javadoc 1m 12s /results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt hadoop-common-project_hadoop-common-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 :green_heart: javadoc 5m 20s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 10m 39s the patch passed
+1 :green_heart: shadedclient 34m 27s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 19m 53s hadoop-common in the patch passed.
+1 :green_heart: unit 226m 33s hadoop-hdfs in the patch passed.
+1 :green_heart: unit 3m 21s hadoop-aws in the patch passed.
+1 :green_heart: unit 2m 55s hadoop-azure in the patch passed.
+1 :green_heart: unit 0m 55s hadoop-aliyun in the patch passed.
+1 :green_heart: asflicense 1m 14s The patch does not generate ASF License warnings.
510m 43s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/31/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6686
Optional Tests dupname asflicense codespell detsecrets xmllint compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle markdownlint
uname Linux dcb10dc97a56 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / a60f7690b7aa230f6852dd3af74b439c6a57f5fe
Default Java Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/31/testReport/
Max. process+thread count 3660 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure hadoop-tools/hadoop-aliyun U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/31/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Jul 27 '24 01:07 hadoop-yetus

javadocs

hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FunctionalIO.java:83: warning: no @param for <R>

checkstyles are all about use of _ in method names, except for one

./hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/wrappedio/impl/TestWrappedStatistics.java:257:    snapshot.setCounter( "c1", 10);:24: '(' is followed by whitespace. [ParenPad]

steveloughran avatar Jul 29 '24 12:07 steveloughran

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 11m 53s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+0 :ok: xmllint 0m 0s xmllint was not available.
+0 :ok: markdownlint 0m 1s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 14 new or modified test files.
_ trunk Compile Tests _
+0 :ok: mvndep 14m 54s Maven dependency ordering for branch
+1 :green_heart: mvninstall 33m 43s trunk passed
+1 :green_heart: compile 17m 23s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: compile 16m 15s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: checkstyle 4m 21s trunk passed
+1 :green_heart: mvnsite 6m 2s trunk passed
+1 :green_heart: javadoc 4m 36s trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 4m 53s trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 9m 51s trunk passed
+1 :green_heart: shadedclient 34m 10s branch has no errors when building and testing our client artifacts.
-0 :warning: patch 34m 37s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 32s Maven dependency ordering for patch
+1 :green_heart: mvninstall 3m 35s the patch passed
+1 :green_heart: compile 16m 56s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javac 16m 56s the patch passed
+1 :green_heart: compile 16m 16s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: javac 16m 16s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 4m 22s /results-checkstyle-root.txt root: The patch generated 60 new + 66 unchanged - 3 fixed = 126 total (was 69)
+1 :green_heart: mvnsite 6m 0s the patch passed
+1 :green_heart: javadoc 4m 44s the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 :green_heart: javadoc 5m 17s the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 :green_heart: spotbugs 10m 42s the patch passed
+1 :green_heart: shadedclient 34m 40s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 :x: unit 19m 53s /patch-unit-hadoop-common-project_hadoop-common.txt hadoop-common in the patch passed.
-1 :x: unit 85m 10s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
-1 :x: unit 0m 56s /patch-unit-hadoop-tools_hadoop-aws.txt hadoop-aws in the patch failed.
-1 :x: unit 0m 45s /patch-unit-hadoop-tools_hadoop-azure.txt hadoop-azure in the patch failed.
-1 :x: unit 0m 44s /patch-unit-hadoop-tools_hadoop-aliyun.txt hadoop-aliyun in the patch failed.
+0 :ok: asflicense 0m 46s ASF License check generated no output?
375m 8s
Reason Tests
Failed junit tests hadoop.util.functional.TestFunctionalIO
hadoop.hdfs.TestHdfsAdmin
hadoop.hdfs.web.TestWebHDFS
hadoop.hdfs.qjournal.TestSecureNNWithQJM
hadoop.hdfs.TestDecommissionWithStripedBackoffMonitor
hadoop.hdfs.TestDistributedFileSystem
hadoop.hdfs.TestBlockTokenWrappingQOP
hadoop.hdfs.TestDFSClientExcludedNodes
hadoop.hdfs.TestErasureCodeBenchmarkThroughput
hadoop.hdfs.TestSecureEncryptionZoneWithKMS
hadoop.hdfs.qjournal.TestNNWithQJM
hadoop.hdfs.TestDFSRollback
hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter
hadoop.hdfs.TestWriteRead
hadoop.hdfs.TestReservedRawPaths
hadoop.hdfs.TestDFSUpgradeFromImage
hadoop.hdfs.web.TestWebHDFSXAttr
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/32/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6686
Optional Tests dupname asflicense codespell detsecrets xmllint compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle markdownlint
uname Linux c220f37dde5d 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / b59846b6d906d3e96fc32c303e4dfb9c74b7fa80
Default Java Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/32/testReport/
Max. process+thread count 2251 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure hadoop-tools/hadoop-aliyun U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/32/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Jul 29 '24 18:07 hadoop-yetus

legitimate failure

 Expected to find '404' but got unexpected exception: java.io.UncheckedIOException: java.io.FileNotFoundException: missing
 at org.apache.hadoop.util.functional.FunctionRaisingIOE.unchecked(FunctionRaisingIOE.java:50)
 at org.apache.hadoop.util.functional.TestFunctionalIO.lambda$testUncheckedFunction$9(TestFunctionalIO.java:105)
 at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:500)
 at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:386)
 at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:455)
 at org.apache.hadoop.util.functional.TestFunctionalIO.testUncheckedFunction(TestFunctionalIO.java:104)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
 at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
 at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.lang.Thread.run(Thread.java:750)
Caused by: java.io.FileNotFoundException: missing
 at org.apache.hadoop.util.functional.TestFunctionalIO.lambda$testUncheckedFunction$8(TestFunctionalIO.java:103)
 at org.apache.hadoop.util.functional.FunctionRaisingIOE.unchecked(FunctionRaisingIOE.java:48)
 ... 18 more

steveloughran avatar Aug 02 '24 14:08 steveloughran

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 11m 59s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 1s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+0 :ok: xmllint 0m 1s xmllint was not available.
+0 :ok: markdownlint 0m 1s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 15 new or modified test files.
_ trunk Compile Tests _
+0 :ok: mvndep 15m 9s Maven dependency ordering for branch
+1 :green_heart: mvninstall 36m 50s trunk passed
+1 :green_heart: compile 18m 46s trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: compile 16m 13s trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: checkstyle 4m 25s trunk passed
+1 :green_heart: mvnsite 5m 58s trunk passed
+1 :green_heart: javadoc 4m 47s trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: javadoc 5m 14s trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: spotbugs 9m 35s trunk passed
+1 :green_heart: shadedclient 34m 54s branch has no errors when building and testing our client artifacts.
-0 :warning: patch 35m 23s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 31s Maven dependency ordering for patch
+1 :green_heart: mvninstall 3m 39s the patch passed
+1 :green_heart: compile 17m 4s the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: javac 17m 4s the patch passed
+1 :green_heart: compile 16m 3s the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: javac 16m 3s the patch passed
-1 :x: blanks 0m 0s /blanks-eol.txt The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 :warning: checkstyle 5m 11s /results-checkstyle-root.txt root: The patch generated 58 new + 66 unchanged - 3 fixed = 124 total (was 69)
+1 :green_heart: mvnsite 5m 44s the patch passed
+1 :green_heart: javadoc 4m 27s the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: javadoc 5m 24s the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: spotbugs 10m 26s the patch passed
+1 :green_heart: shadedclient 36m 40s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 19m 28s hadoop-common in the patch passed.
+1 :green_heart: unit 226m 10s hadoop-hdfs in the patch passed.
+1 :green_heart: unit 3m 4s hadoop-aws in the patch passed.
+1 :green_heart: unit 2m 54s hadoop-azure in the patch passed.
+1 :green_heart: unit 0m 45s hadoop-aliyun in the patch passed.
+1 :green_heart: asflicense 1m 9s The patch does not generate ASF License warnings.
528m 6s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/33/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6686
Optional Tests dupname asflicense codespell detsecrets xmllint compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle markdownlint
uname Linux e67f494e0409 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 3fe9cdb1a164256487f5fcaebc15dec23a53111a
Default Java Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/33/testReport/
Max. process+thread count 4028 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure hadoop-tools/hadoop-aliyun U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/33/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 07 '24 22:08 hadoop-yetus

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 30s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+0 :ok: xmllint 0m 0s xmllint was not available.
+0 :ok: markdownlint 0m 1s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 15 new or modified test files.
_ trunk Compile Tests _
+0 :ok: mvndep 15m 0s Maven dependency ordering for branch
+1 :green_heart: mvninstall 35m 46s trunk passed
+1 :green_heart: compile 19m 15s trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: compile 17m 24s trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: checkstyle 4m 45s trunk passed
+1 :green_heart: mvnsite 5m 32s trunk passed
+1 :green_heart: javadoc 4m 19s trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: javadoc 4m 51s trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: spotbugs 9m 16s trunk passed
+1 :green_heart: shadedclient 40m 31s branch has no errors when building and testing our client artifacts.
-0 :warning: patch 40m 58s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 32s Maven dependency ordering for patch
+1 :green_heart: mvninstall 3m 52s the patch passed
+1 :green_heart: compile 20m 17s the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: javac 20m 17s the patch passed
+1 :green_heart: compile 18m 24s the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: javac 18m 24s the patch passed
-1 :x: blanks 0m 0s /blanks-eol.txt The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 :warning: checkstyle 5m 2s /results-checkstyle-root.txt root: The patch generated 58 new + 66 unchanged - 3 fixed = 124 total (was 69)
+1 :green_heart: mvnsite 5m 59s the patch passed
+1 :green_heart: javadoc 4m 27s the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: javadoc 4m 59s the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: spotbugs 11m 19s the patch passed
+1 :green_heart: shadedclient 38m 39s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 20m 26s hadoop-common in the patch passed.
+1 :green_heart: unit 227m 59s hadoop-hdfs in the patch passed.
+1 :green_heart: unit 3m 14s hadoop-aws in the patch passed.
+1 :green_heart: unit 2m 54s hadoop-azure in the patch passed.
+1 :green_heart: unit 0m 55s hadoop-aliyun in the patch passed.
+1 :green_heart: asflicense 1m 13s The patch does not generate ASF License warnings.
532m 59s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/34/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6686
Optional Tests dupname asflicense codespell detsecrets xmllint compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle markdownlint
uname Linux 8d5fb1d05a35 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 76b0afce2ea7be2fff23f876500d03d17d7529be
Default Java Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/34/testReport/
Max. process+thread count 3385 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure hadoop-tools/hadoop-aliyun U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/34/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 08 '24 01:08 hadoop-yetus

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 11m 39s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 1s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+0 :ok: xmllint 0m 1s xmllint was not available.
+0 :ok: markdownlint 0m 0s markdownlint was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: test4tests 0m 0s The patch appears to include 15 new or modified test files.
_ trunk Compile Tests _
+0 :ok: mvndep 15m 10s Maven dependency ordering for branch
+1 :green_heart: mvninstall 32m 59s trunk passed
+1 :green_heart: compile 17m 50s trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: compile 16m 7s trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: checkstyle 4m 21s trunk passed
+1 :green_heart: mvnsite 5m 58s trunk passed
+1 :green_heart: javadoc 4m 52s trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: javadoc 5m 9s trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: spotbugs 9m 41s trunk passed
+1 :green_heart: shadedclient 34m 41s branch has no errors when building and testing our client artifacts.
-0 :warning: patch 35m 8s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 33s Maven dependency ordering for patch
+1 :green_heart: mvninstall 3m 39s the patch passed
+1 :green_heart: compile 16m 59s the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: javac 16m 59s the patch passed
+1 :green_heart: compile 16m 20s the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: javac 16m 20s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 4m 26s /results-checkstyle-root.txt root: The patch generated 58 new + 66 unchanged - 3 fixed = 124 total (was 69)
+1 :green_heart: mvnsite 6m 4s the patch passed
+1 :green_heart: javadoc 4m 44s the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 :green_heart: javadoc 5m 22s the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 :green_heart: spotbugs 10m 36s the patch passed
+1 :green_heart: shadedclient 34m 50s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 :green_heart: unit 19m 54s hadoop-common in the patch passed.
+1 :green_heart: unit 226m 42s hadoop-hdfs in the patch passed.
+1 :green_heart: unit 3m 21s hadoop-aws in the patch passed.
+1 :green_heart: unit 2m 56s hadoop-azure in the patch passed.
+1 :green_heart: unit 0m 54s hadoop-aliyun in the patch passed.
+1 :green_heart: asflicense 1m 14s The patch does not generate ASF License warnings.
522m 37s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/35/artifact/out/Dockerfile
GITHUB PR https://github.com/apache/hadoop/pull/6686
Optional Tests dupname asflicense codespell detsecrets xmllint compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle markdownlint
uname Linux 2ebdeb4da928 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 20d385c3d2c48b98bb0fecb6a1d4672606bc785a
Default Java Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/35/testReport/
Max. process+thread count 3527 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure hadoop-tools/hadoop-aliyun U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6686/35/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus avatar Aug 13 '24 03:08 hadoop-yetus

All checkstyles are from underscores; I tried to set up a style rule to disable this but it didn't work right as there's no checkstyle overrides in hadoop-common right now

steveloughran avatar Aug 13 '24 10:08 steveloughran

fs.capability.virtual.block.locations

it's to say "this fs makes up block locations". it means that cost of looking up block locations is a lot less (no remote calls) and you don't really need to schedule work elsewhere.

now that hasPathCapability() is being exported to legacy code, I just felt this would be useful. Currently things look for the default (host == localhost) and go from there -but they only get to do that after the looup

steveloughran avatar Aug 14 '24 10:08 steveloughran

Hi @steveloughran Please check my comment on the merged commit. https://github.com/apache/hadoop/commit/55a576906dda00a04be1f13ed7f61e32ef73e309#r147058497

Sorry I should have added that comment her itself. Thanks

anujmodi2021 avatar Sep 23 '24 08:09 anujmodi2021