hadoop
hadoop copied to clipboard
Published
20 hours ago •
apache
apache
HADOOP-18257. Analyze S3A Audit Logs
Description of PR
HADOOP-18257. Analyze S3A Audit Logs
This is #6000 rebased and squashed with a couple of changes on top.
- tool is invoked through hadoop s3guard command
- which can now also be invoked as "hadoop s3a"! (not documented, maybe we should isolate?)
- tests are improved.
- OperationDuration implements DurationTracker for bit more completeness
TODO
- split out the record parsing into a hadoop MR record read/write
- so support large scale parsing
- cli tool just glues that together either for small parallelised extraction or for aggregation to one file.
- but a bulk job would work with a larger dataset
How was this patch tested?
new unit test cases.
For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
- [ ] If applicable, have you updated the
LICENSE,LICENSE-binary,NOTICE-binaryfiles?
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 6m 38s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | xmllint | 0m 0s | xmllint was not available. | |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +0 :ok: | mvndep | 14m 21s | Maven dependency ordering for branch | |
| +1 :green_heart: | mvninstall | 20m 18s | trunk passed | |
| +1 :green_heart: | compile | 8m 38s | trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 | |
| +1 :green_heart: | compile | 8m 2s | trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 | |
| +1 :green_heart: | checkstyle | 2m 3s | trunk passed | |
| +1 :green_heart: | mvnsite | 14m 36s | trunk passed | |
| +1 :green_heart: | javadoc | 4m 51s | trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 | |
| +1 :green_heart: | javadoc | 4m 53s | trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 | |
| -1 :x: | spotbugs | 17m 6s | /branch-spotbugs-root-warnings.html | root in trunk has 2 extant spotbugs warnings. |
| +1 :green_heart: | shadedclient | 36m 36s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for patch | |
| +1 :green_heart: | mvninstall | 18m 35s | the patch passed | |
| +1 :green_heart: | compile | 8m 24s | the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 | |
| +1 :green_heart: | javac | 8m 24s | the patch passed | |
| +1 :green_heart: | compile | 8m 6s | the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 | |
| +1 :green_heart: | javac | 8m 6s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 2m 0s | /results-checkstyle-root.txt | root: The patch generated 7 new + 2 unchanged - 0 fixed = 9 total (was 2) |
| +1 :green_heart: | mvnsite | 8m 41s | the patch passed | |
| +1 :green_heart: | shellcheck | 0m 1s | No new issues. | |
| +1 :green_heart: | javadoc | 4m 45s | the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 | |
| +1 :green_heart: | javadoc | 4m 54s | the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 | |
| -1 :x: | spotbugs | 0m 46s | /new-spotbugs-hadoop-tools_hadoop-aws.html | hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| -1 :x: | spotbugs | 17m 27s | /new-spotbugs-root.html | root generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) |
| +1 :green_heart: | shadedclient | 37m 11s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 645m 36s | /patch-unit-root.txt | root in the patch passed. |
| +1 :green_heart: | asflicense | 1m 4s | The patch does not generate ASF License warnings. | |
| 868m 4s |
| Reason | Tests |
|---|---|
| SpotBugs | module:hadoop-tools/hadoop-aws |
| Found reliance on default encoding in org.apache.hadoop.fs.s3a.audit.AuditTool.run(String[], PrintStream):in org.apache.hadoop.fs.s3a.audit.AuditTool.run(String[], PrintStream): new java.io.PrintWriter(OutputStream) At AuditTool.java:[line 122] | |
| SpotBugs | module:root |
| Found reliance on default encoding in org.apache.hadoop.fs.s3a.audit.AuditTool.run(String[], PrintStream):in org.apache.hadoop.fs.s3a.audit.AuditTool.run(String[], PrintStream): new java.io.PrintWriter(OutputStream) At AuditTool.java:[line 122] | |
| Failed junit tests | hadoop.hdfs.server.blockmanagement.TestDatanodeManager |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6901 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs |
| uname | Linux 346db37e4260 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f3e1bd5af4ea6bc9fe84156d9271d8569e708988 |
| Default Java | Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/1/testReport/ |
| Max. process+thread count | 4268 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws . U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 19s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | xmllint | 0m 0s | xmllint was not available. | |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +0 :ok: | mvndep | 15m 25s | Maven dependency ordering for branch | |
| +1 :green_heart: | mvninstall | 20m 22s | trunk passed | |
| +1 :green_heart: | compile | 9m 4s | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| +1 :green_heart: | compile | 8m 22s | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 | |
| +1 :green_heart: | checkstyle | 2m 17s | trunk passed | |
| +1 :green_heart: | mvnsite | 18m 35s | trunk passed | |
| +1 :green_heart: | javadoc | 5m 3s | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| +1 :green_heart: | javadoc | 5m 0s | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 | |
| -1 :x: | spotbugs | 17m 35s | /branch-spotbugs-root-warnings.html | root in trunk has 2 extant spotbugs warnings. |
| +1 :green_heart: | shadedclient | 37m 33s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for patch | |
| +1 :green_heart: | mvninstall | 18m 53s | the patch passed | |
| +1 :green_heart: | compile | 8m 41s | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| +1 :green_heart: | javac | 8m 41s | the patch passed | |
| +1 :green_heart: | compile | 8m 24s | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 | |
| +1 :green_heart: | javac | 8m 24s | the patch passed | |
| -1 :x: | blanks | 0m 0s | /blanks-eol.txt | The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 2m 2s | /results-checkstyle-root.txt | root: The patch generated 2 new + 2 unchanged - 0 fixed = 4 total (was 2) |
| +1 :green_heart: | mvnsite | 11m 36s | the patch passed | |
| +1 :green_heart: | shellcheck | 0m 0s | No new issues. | |
| +1 :green_heart: | javadoc | 4m 54s | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| +1 :green_heart: | javadoc | 5m 5s | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 | |
| +1 :green_heart: | spotbugs | 19m 48s | the patch passed | |
| +1 :green_heart: | shadedclient | 37m 41s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 639m 13s | /patch-unit-root.txt | root in the patch passed. |
| +1 :green_heart: | asflicense | 0m 56s | The patch does not generate ASF License warnings. | |
| 867m 41s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2 |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6901 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs |
| uname | Linux 913d0845eac3 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / fe0322ab711a250ca5e2d6a83c332812a5031460 |
| Default Java | Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/2/testReport/ |
| Max. process+thread count | 4010 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws . U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 21s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | xmllint | 0m 0s | xmllint was not available. | |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +0 :ok: | mvndep | 15m 53s | Maven dependency ordering for branch | |
| +1 :green_heart: | mvninstall | 21m 24s | trunk passed | |
| +1 :green_heart: | compile | 10m 1s | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| +1 :green_heart: | compile | 9m 13s | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 | |
| +1 :green_heart: | checkstyle | 2m 25s | trunk passed | |
| +1 :green_heart: | mvnsite | 16m 47s | trunk passed | |
| +1 :green_heart: | javadoc | 5m 19s | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| +1 :green_heart: | javadoc | 5m 3s | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 | |
| -1 :x: | spotbugs | 18m 8s | /branch-spotbugs-root-warnings.html | root in trunk has 2 extant spotbugs warnings. |
| +1 :green_heart: | shadedclient | 40m 7s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for patch | |
| +1 :green_heart: | mvninstall | 21m 50s | the patch passed | |
| +1 :green_heart: | compile | 9m 56s | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| +1 :green_heart: | javac | 9m 56s | the patch passed | |
| +1 :green_heart: | compile | 9m 27s | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 | |
| +1 :green_heart: | javac | 9m 27s | the patch passed | |
| -1 :x: | blanks | 0m 0s | /blanks-eol.txt | The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 2m 15s | /results-checkstyle-root.txt | root: The patch generated 8 new + 2 unchanged - 0 fixed = 10 total (was 2) |
| +1 :green_heart: | mvnsite | 9m 49s | the patch passed | |
| +1 :green_heart: | shellcheck | 0m 0s | No new issues. | |
| +1 :green_heart: | javadoc | 4m 24s | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| -1 :x: | javadoc | 4m 33s | /patch-javadoc-root-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt | root in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05. |
| -1 :x: | spotbugs | 0m 49s | /new-spotbugs-hadoop-tools_hadoop-aws.html | hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| -1 :x: | spotbugs | 17m 55s | /new-spotbugs-root.html | root generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) |
| +1 :green_heart: | shadedclient | 37m 57s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 639m 16s | /patch-unit-root.txt | root in the patch passed. |
| +1 :green_heart: | asflicense | 1m 8s | The patch does not generate ASF License warnings. | |
| 875m 29s |
| Reason | Tests |
|---|---|
| SpotBugs | module:hadoop-tools/hadoop-aws |
| org.apache.hadoop.fs.s3a.audit.mapreduce.S3AAuditLogMergerAndParser.FIELDS_OF_TYPE_LONG is a mutable collection which should be package protected At S3AAuditLogMergerAndParser.java:which should be package protected At S3AAuditLogMergerAndParser.java:[line 76] | |
| SpotBugs | module:root |
| org.apache.hadoop.fs.s3a.audit.mapreduce.S3AAuditLogMergerAndParser.FIELDS_OF_TYPE_LONG is a mutable collection which should be package protected At S3AAuditLogMergerAndParser.java:which should be package protected At S3AAuditLogMergerAndParser.java:[line 76] | |
| Failed junit tests | hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2 |
| hadoop.fs.s3a.audit.TestAuditTool |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6901 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs |
| uname | Linux 48468eb1eff9 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 73035635ee1fce6103e14cfbb10d3bcbba01d5e5 |
| Default Java | Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/3/testReport/ |
| Max. process+thread count | 4787 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws . U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 20s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | xmllint | 0m 0s | xmllint was not available. | |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +0 :ok: | mvndep | 15m 34s | Maven dependency ordering for branch | |
| +1 :green_heart: | mvninstall | 19m 19s | trunk passed | |
| +1 :green_heart: | compile | 8m 57s | trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 | |
| +1 :green_heart: | compile | 8m 22s | trunk passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga | |
| +1 :green_heart: | checkstyle | 2m 4s | trunk passed | |
| +1 :green_heart: | mvnsite | 14m 58s | trunk passed | |
| +1 :green_heart: | javadoc | 5m 20s | trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 | |
| +1 :green_heart: | javadoc | 4m 58s | trunk passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga | |
| +1 :green_heart: | spotbugs | 19m 17s | trunk passed | |
| +1 :green_heart: | shadedclient | 37m 30s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for patch | |
| -1 :x: | mvninstall | 0m 14s | /patch-mvninstall-hadoop-tools_hadoop-aws.txt | hadoop-aws in the patch failed. |
| -1 :x: | mvninstall | 18m 5s | /patch-mvninstall-root.txt | root in the patch failed. |
| -1 :x: | compile | 8m 23s | /patch-compile-root-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt | root in the patch failed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | javac | 8m 23s | /patch-compile-root-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt | root in the patch failed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | compile | 8m 10s | /patch-compile-root-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt | root in the patch failed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga. |
| -1 :x: | javac | 8m 10s | /patch-compile-root-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt | root in the patch failed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga. |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 2m 31s | /results-checkstyle-root.txt | root: The patch generated 6 new + 2 unchanged - 0 fixed = 8 total (was 2) |
| -1 :x: | mvnsite | 2m 56s | /patch-mvnsite-root.txt | root in the patch failed. |
| +1 :green_heart: | shellcheck | 0m 0s | No new issues. | |
| -1 :x: | javadoc | 4m 55s | /patch-javadoc-root-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt | root in the patch failed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | javadoc | 4m 33s | /patch-javadoc-root-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt | root in the patch failed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga. |
| -1 :x: | spotbugs | 0m 21s | /patch-spotbugs-hadoop-tools_hadoop-aws.txt | hadoop-aws in the patch failed. |
| -1 :x: | spotbugs | 15m 50s | /patch-spotbugs-root.txt | root in the patch failed. |
| +1 :green_heart: | shadedclient | 21m 25s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 647m 48s | /patch-unit-root.txt | root in the patch failed. |
| +1 :green_heart: | asflicense | 1m 6s | The patch does not generate ASF License warnings. | |
| 860m 43s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy |
| hadoop.hdfs.TestDecommissionWithBackoffMonitor | |
| hadoop.hdfs.TestDecommission |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6901 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs |
| uname | Linux a5a51b1cb4b9 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / fb2a10e46ca08165292536914d54a984259a3564 |
| Default Java | Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/4/testReport/ |
| Max. process+thread count | 4350 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws . U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 18s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | xmllint | 0m 0s | xmllint was not available. | |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +0 :ok: | mvndep | 14m 38s | Maven dependency ordering for branch | |
| +1 :green_heart: | mvninstall | 20m 40s | trunk passed | |
| +1 :green_heart: | compile | 10m 36s | trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 | |
| +1 :green_heart: | compile | 8m 49s | trunk passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga | |
| +1 :green_heart: | checkstyle | 2m 20s | trunk passed | |
| +1 :green_heart: | mvnsite | 14m 54s | trunk passed | |
| +1 :green_heart: | javadoc | 5m 24s | trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 | |
| +1 :green_heart: | javadoc | 4m 53s | trunk passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga | |
| +1 :green_heart: | spotbugs | 19m 18s | trunk passed | |
| +1 :green_heart: | shadedclient | 37m 29s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for patch | |
| +1 :green_heart: | mvninstall | 18m 46s | the patch passed | |
| +1 :green_heart: | compile | 8m 43s | the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 | |
| +1 :green_heart: | javac | 8m 43s | the patch passed | |
| +1 :green_heart: | compile | 8m 30s | the patch passed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga | |
| +1 :green_heart: | javac | 8m 30s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 2m 4s | /results-checkstyle-root.txt | root: The patch generated 6 new + 2 unchanged - 0 fixed = 8 total (was 2) |
| +1 :green_heart: | mvnsite | 9m 58s | the patch passed | |
| +1 :green_heart: | shellcheck | 0m 1s | No new issues. | |
| -1 :x: | javadoc | 4m 48s | /patch-javadoc-root-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt | root in the patch failed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | javadoc | 4m 32s | /patch-javadoc-root-jdkPrivateBuild-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga.txt | root in the patch failed with JDK Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga. |
| -1 :x: | spotbugs | 0m 49s | /new-spotbugs-hadoop-tools_hadoop-aws.html | hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| -1 :x: | spotbugs | 17m 32s | /new-spotbugs-root.html | root generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | shadedclient | 37m 33s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 647m 39s | /patch-unit-root.txt | root in the patch passed. |
| +1 :green_heart: | asflicense | 0m 52s | The patch does not generate ASF License warnings. | |
| 871m 52s |
| Reason | Tests |
|---|---|
| SpotBugs | module:hadoop-tools/hadoop-aws |
| org.apache.hadoop.fs.s3a.audit.mapreduce.S3AAuditLogMergerAndParser.FIELDS_OF_TYPE_LONG is a mutable collection which should be package protected At S3AAuditLogMergerAndParser.java:which should be package protected At S3AAuditLogMergerAndParser.java:[line 76] | |
| SpotBugs | module:root |
| org.apache.hadoop.fs.s3a.audit.mapreduce.S3AAuditLogMergerAndParser.FIELDS_OF_TYPE_LONG is a mutable collection which should be package protected At S3AAuditLogMergerAndParser.java:which should be package protected At S3AAuditLogMergerAndParser.java:[line 76] | |
| Failed junit tests | hadoop.hdfs.TestDecommissionWithBackoffMonitor |
| hadoop.hdfs.TestDecommission | |
| hadoop.fs.s3a.audit.TestAuditTool |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6901 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs |
| uname | Linux 81b7addc1e9b 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e012694d147e3bdde47eb54c3d2668abfbb90d2f |
| Default Java | Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_432-8u432-ga~us1-0ubuntu2~20.04-ga |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/5/testReport/ |
| Max. process+thread count | 4598 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws . U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 20s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | xmllint | 0m 0s | xmllint was not available. | |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +0 :ok: | mvndep | 7m 8s | Maven dependency ordering for branch | |
| +1 :green_heart: | mvninstall | 18m 51s | trunk passed | |
| +1 :green_heart: | compile | 8m 15s | trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 | |
| +1 :green_heart: | compile | 7m 21s | trunk passed with JDK Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 | |
| +1 :green_heart: | checkstyle | 1m 55s | trunk passed | |
| +1 :green_heart: | mvnsite | 14m 5s | trunk passed | |
| +1 :green_heart: | javadoc | 6m 5s | trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 | |
| +1 :green_heart: | javadoc | 4m 46s | trunk passed with JDK Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 | |
| +1 :green_heart: | spotbugs | 19m 27s | trunk passed | |
| +1 :green_heart: | shadedclient | 42m 11s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for patch | |
| +1 :green_heart: | mvninstall | 20m 38s | the patch passed | |
| +1 :green_heart: | compile | 8m 57s | the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 | |
| +1 :green_heart: | javac | 8m 57s | the patch passed | |
| +1 :green_heart: | compile | 7m 37s | the patch passed with JDK Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 | |
| +1 :green_heart: | javac | 7m 37s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 1m 54s | /results-checkstyle-root.txt | root: The patch generated 6 new + 2 unchanged - 0 fixed = 8 total (was 2) |
| +1 :green_heart: | mvnsite | 12m 42s | the patch passed | |
| +1 :green_heart: | shellcheck | 0m 0s | No new issues. | |
| -1 :x: | javadoc | 4m 33s | /patch-javadoc-root-jdkUbuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04.txt | root in the patch failed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04. |
| -1 :x: | javadoc | 4m 16s | /patch-javadoc-root-jdkPrivateBuild-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06.txt | root in the patch failed with JDK Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06. |
| -1 :x: | spotbugs | 0m 45s | /new-spotbugs-hadoop-tools_hadoop-aws.html | hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| -1 :x: | spotbugs | 18m 34s | /new-spotbugs-root.html | root generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | shadedclient | 39m 38s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 493m 24s | /patch-unit-root.txt | root in the patch passed. |
| +1 :green_heart: | asflicense | 1m 0s | The patch does not generate ASF License warnings. | |
| 713m 36s |
| Reason | Tests |
|---|---|
| SpotBugs | module:hadoop-tools/hadoop-aws |
| org.apache.hadoop.fs.s3a.audit.mapreduce.S3AAuditLogMergerAndParser.FIELDS_OF_TYPE_LONG is a mutable collection which should be package protected At S3AAuditLogMergerAndParser.java:which should be package protected At S3AAuditLogMergerAndParser.java:[line 76] | |
| SpotBugs | module:root |
| org.apache.hadoop.fs.s3a.audit.mapreduce.S3AAuditLogMergerAndParser.FIELDS_OF_TYPE_LONG is a mutable collection which should be package protected At S3AAuditLogMergerAndParser.java:which should be package protected At S3AAuditLogMergerAndParser.java:[line 76] | |
| Failed junit tests | hadoop.fs.s3a.audit.TestAuditTool |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6901 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs |
| uname | Linux 12bcbce6ac71 5.15.0-136-generic #147-Ubuntu SMP Sat Mar 15 15:53:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8cd6bec84c4b1d786a78b8927af7a50b39af9f0e |
| Default Java | Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/6/testReport/ |
| Max. process+thread count | 2975 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws . U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
hasn't adaped to the 11.4 upgrade as it was using avro ser/deser as part of the workflow, and that can't be done with the class restrictions.
We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again. Thanks all for your contribution.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}