hadoop
hadoop copied to clipboard
Published
20 hours ago •
apache
apache
HADOOP-19083. hadoop binaries aws sdk unless -Pawssdk builds
New profile awssdk must be set for hadoop-tools-dist to include the AWS SDK (so cuts binary size in half)
- This is not yet documented
- no attempt to be clever in LICENSE-binary and remove the declaration.
For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
- [ ] If applicable, have you updated the
LICENSE,LICENSE-binary,NOTICE-binaryfiles?
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 1s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
| +0 :ok: | xmllint | 0m 1s | xmllint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 44m 17s | trunk passed | |
| +1 :green_heart: | compile | 0m 25s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 0m 24s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | mvnsite | 0m 30s | trunk passed | |
| +1 :green_heart: | javadoc | 0m 30s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 24s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | shadedclient | 78m 48s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 0m 17s | the patch passed | |
| +1 :green_heart: | compile | 0m 14s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javac | 0m 14s | the patch passed | |
| +1 :green_heart: | compile | 0m 15s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | javac | 0m 15s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | mvnsite | 0m 17s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 16s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 15s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | shadedclient | 33m 51s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 0m 19s | hadoop-tools-dist in the patch passed. | |
| +1 :green_heart: | asflicense | 0m 39s | The patch does not generate ASF License warnings. | |
| 119m 20s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6570/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6570 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint |
| uname | Linux d60acba26022 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 31f9a71b8c85ff79f9aa2ac5754686e75741b58a |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6570/1/testReport/ |
| Max. process+thread count | 691 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-tools-dist U: hadoop-tools/hadoop-tools-dist |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6570/1/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
we may want to merge this for anyone doing lightweight local builds, but for our own releases it is better just to make some lean tarballs where we strip out the bundle.jar file, just as we swap over the arm64 binaries.
let me rebase.
I was thinking it'd be better to just generate a .tar.gz with the aws sdk stripped from it. that way: one build, identical binaries
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 50s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | |
| +0 :ok: | xmllint | 0m 0s | xmllint was not available. | |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | |
| _ trunk Compile Tests _ | ||||
| +0 :ok: | mvndep | 14m 21s | Maven dependency ordering for branch | |
| +1 :green_heart: | mvninstall | 36m 40s | trunk passed | |
| +1 :green_heart: | compile | 19m 35s | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| +1 :green_heart: | compile | 17m 49s | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 | |
| +1 :green_heart: | checkstyle | 4m 57s | trunk passed | |
| +1 :green_heart: | mvnsite | 27m 1s | trunk passed | |
| +1 :green_heart: | javadoc | 9m 11s | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| +1 :green_heart: | javadoc | 8m 0s | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 | |
| +0 :ok: | spotbugs | 0m 20s | branch/hadoop-tools/hadoop-tools-dist no spotbugs output file (spotbugsXml.xml) | |
| -1 :x: | spotbugs | 32m 49s | /branch-spotbugs-root-warnings.html | root in trunk has 2 extant spotbugs warnings. |
| +1 :green_heart: | shadedclient | 71m 9s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 31s | Maven dependency ordering for patch | |
| +1 :green_heart: | mvninstall | 35m 39s | the patch passed | |
| +1 :green_heart: | compile | 19m 7s | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| +1 :green_heart: | javac | 19m 7s | the patch passed | |
| +1 :green_heart: | compile | 17m 43s | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 | |
| +1 :green_heart: | javac | 17m 43s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 4m 42s | the patch passed | |
| +1 :green_heart: | mvnsite | 20m 16s | the patch passed | |
| +1 :green_heart: | shellcheck | 0m 0s | No new issues. | |
| +1 :green_heart: | javadoc | 9m 5s | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| +1 :green_heart: | javadoc | 7m 55s | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 | |
| +0 :ok: | spotbugs | 0m 20s | hadoop-tools/hadoop-tools-dist has no data from spotbugs | |
| +1 :green_heart: | shadedclient | 72m 12s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 818m 46s | root in the patch passed. | |
| +1 :green_heart: | asflicense | 1m 41s | The patch does not generate ASF License warnings. | |
| 1226m 1s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6570/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6570 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle markdownlint xmllint shellcheck shelldocs |
| uname | Linux 0fcb0c404a1c 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 092c7806c2f971dbc9a9bbdc5b56c5b201a74bdf |
| Default Java | Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6570/3/testReport/ |
| Max. process+thread count | 3582 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws hadoop-tools/hadoop-tools-dist . U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6570/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 34s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | |
| +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | |
| +0 :ok: | xmllint | 0m 0s | xmllint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | |
| _ trunk Compile Tests _ | ||||
| +0 :ok: | mvndep | 15m 10s | Maven dependency ordering for branch | |
| +1 :green_heart: | mvninstall | 36m 40s | trunk passed | |
| +1 :green_heart: | compile | 19m 12s | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| +1 :green_heart: | compile | 17m 25s | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 | |
| +1 :green_heart: | checkstyle | 4m 57s | trunk passed | |
| +1 :green_heart: | mvnsite | 26m 6s | trunk passed | |
| +1 :green_heart: | javadoc | 9m 11s | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| +1 :green_heart: | javadoc | 8m 18s | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 | |
| +0 :ok: | spotbugs | 0m 22s | branch/hadoop-tools/hadoop-tools-dist no spotbugs output file (spotbugsXml.xml) | |
| -1 :x: | spotbugs | 33m 11s | /branch-spotbugs-root-warnings.html | root in trunk has 2 extant spotbugs warnings. |
| +1 :green_heart: | shadedclient | 68m 1s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for patch | |
| +1 :green_heart: | mvninstall | 35m 9s | the patch passed | |
| +1 :green_heart: | compile | 18m 6s | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| +1 :green_heart: | javac | 18m 6s | the patch passed | |
| +1 :green_heart: | compile | 17m 19s | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 | |
| +1 :green_heart: | javac | 17m 19s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 4m 41s | the patch passed | |
| +1 :green_heart: | mvnsite | 21m 6s | the patch passed | |
| +1 :green_heart: | shellcheck | 0m 1s | No new issues. | |
| +1 :green_heart: | javadoc | 9m 9s | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 | |
| +1 :green_heart: | javadoc | 8m 26s | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 | |
| +0 :ok: | spotbugs | 0m 22s | hadoop-tools/hadoop-tools-dist has no data from spotbugs | |
| +1 :green_heart: | shadedclient | 72m 9s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 764m 16s | root in the patch passed. | |
| +1 :green_heart: | asflicense | 1m 39s | The patch does not generate ASF License warnings. | |
| 1167m 15s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6570/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6570 |
| Optional Tests | dupname asflicense codespell detsecrets shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle markdownlint xmllint |
| uname | Linux e508c099995a 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 268654b4ace577608e9da10f4a6a25c64816432b |
| Default Java | Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6570/4/testReport/ |
| Max. process+thread count | 3983 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws hadoop-tools/hadoop-tools-dist . U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6570/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.