hadoop
hadoop copied to clipboard
Published
20 hours ago •
apache
apache
HADOOP-19036. Standardize Maven Initialization Across Operating Systems.
Description of PR
JIRA: HADOOP-19036. Standardize Maven Initialization Across Operating Systems.
The differences in initializing Maven for various operating systems in the build scripts are as follows:
- For Ubuntu and Debian, Maven is installed using the yum repository.
- For CentOS 7 and CentOS 8, Maven is downloaded remotely.
- The Maven version used on Windows is inconsistent with other operating systems.
How was this patch tested?
For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
- [ ] If applicable, have you updated the
LICENSE,LICENSE-binary,NOTICE-binaryfiles?
This PR will take some time to be verified. I have currently tested Ubuntu and Centos7 and it is in line with expectations.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 0s | Docker mode activated. | |
| -1 :x: | docker | 2m 45s | Docker failed to build run-specific yetus/hadoop:tp-16504}. |
| Subsystem | Report/Notes |
|---|---|
| GITHUB PR | https://github.com/apache/hadoop/pull/6444 |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6444/1/console |
| versions | git=2.34.1 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 0s | Docker mode activated. | |
| -1 :x: | docker | 0m 2s | Docker failed to build run-specific yetus/hadoop:tp-30132}. |
| Subsystem | Report/Notes |
|---|---|
| GITHUB PR | https://github.com/apache/hadoop/pull/6444 |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6444/2/console |
| versions | git=2.34.1 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
This pr requires HADOOP-19034. Fix Download Maven Url Not Found. Be merged first. Because the address for downloading maven is not available.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 18m 58s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | hadolint | 0m 0s | hadolint was not available. | |
| +0 :ok: | shellcheck | 0m 0s | Shellcheck was not available. | |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | |
| +0 :ok: | jsonlint | 0m 0s | jsonlint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| _ trunk Compile Tests _ | ||||
| +0 :ok: | mvndep | 16m 45s | Maven dependency ordering for branch | |
| +1 :green_heart: | shadedclient | 36m 58s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | shadedclient | 17m 49s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | asflicense | 0m 23s | The patch does not generate ASF License warnings. | |
| 76m 57s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6444/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6444 |
| Optional Tests | dupname asflicense codespell detsecrets hadolint shellcheck shelldocs jsonlint |
| uname | Linux 47f3041735d5 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0df4e80eda97cbdcbf65c3b15ff3a38d989a4b9e |
| Max. process+thread count | 720 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6444/3/console |
| versions | git=2.9.5 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 11m 29s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | hadolint | 0m 0s | hadolint was not available. | |
| +0 :ok: | shellcheck | 0m 0s | Shellcheck was not available. | |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | |
| +0 :ok: | jsonlint | 0m 0s | jsonlint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| _ trunk Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for branch | |
| +1 :green_heart: | shadedclient | 18m 15s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for patch | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | shadedclient | 17m 52s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | |
| 51m 7s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6444/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6444 |
| Optional Tests | dupname asflicense codespell detsecrets hadolint shellcheck shelldocs jsonlint |
| uname | Linux ea07bf1d57f0 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0df4e80eda97cbdcbf65c3b15ff3a38d989a4b9e |
| Max. process+thread count | 566 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6444/3/console |
| versions | git=2.27.0 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 0s | Docker mode activated. | |
| -1 :x: | docker | 1m 18s | Docker failed to build run-specific yetus/hadoop:tp-2527}. |
| Subsystem | Report/Notes |
|---|---|
| GITHUB PR | https://github.com/apache/hadoop/pull/6444 |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6444/3/console |
| versions | git=2.34.1 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
any specific reason to downgrade the maven version?
@steveloughran Thank you for your message!
Downgrading Maven is not due to any issues encountered during compilation, but rather because Maven 3.6.3 is used by other operating systems and it would be better for us to ensure version consistency.
While preparing for the 3.4.0 release, I noticed that the dev-support includes a variety of operating systems such as CentOS 7, CentOS 8, Ubuntu, Debian, Windows, and more.
These operating systems have different ways of initializing Maven, as shown in the following table:
| Operating System | Maven version | Initialization |
|---|---|---|
| Centos7 | 3.6.3 | Download |
| Centos8 | 3.6.3 | Download |
| Ubuntu | 3.6.3 | Yum |
| Debian | 3.6.3 | Yum |
| Windows | 3.8.6 | Download |
My personal opinion is to keep the Maven initialization method and version consistent. However, I would also like to hear the thoughts of other members.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 35m 13s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | hadolint | 0m 0s | hadolint was not available. | |
| +0 :ok: | shellcheck | 0m 0s | Shellcheck was not available. | |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | |
| +0 :ok: | jsonlint | 0m 0s | jsonlint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| _ trunk Compile Tests _ | ||||
| +0 :ok: | mvndep | 14m 35s | Maven dependency ordering for branch | |
| +1 :green_heart: | shadedclient | 47m 59s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 31s | Maven dependency ordering for patch | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | shadedclient | 31m 9s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | |
| 119m 25s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6444/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6444 |
| Optional Tests | dupname asflicense codespell detsecrets hadolint shellcheck shelldocs jsonlint |
| uname | Linux 69654d8c4839 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 05912353a99aa280a785eefe7ae197b54373dac9 |
| Max. process+thread count | 559 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6444/4/console |
| versions | git=2.9.5 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 20m 11s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | hadolint | 0m 0s | hadolint was not available. | |
| +0 :ok: | shellcheck | 0m 0s | Shellcheck was not available. | |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | |
| +0 :ok: | jsonlint | 0m 0s | jsonlint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| _ trunk Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 39s | Maven dependency ordering for branch | |
| +1 :green_heart: | shadedclient | 32m 6s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 39s | Maven dependency ordering for patch | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | shadedclient | 31m 35s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | asflicense | 0m 44s | The patch does not generate ASF License warnings. | |
| 89m 19s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6444/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6444 |
| Optional Tests | dupname asflicense codespell detsecrets hadolint shellcheck shelldocs jsonlint |
| uname | Linux 8d3b86569cf9 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 05912353a99aa280a785eefe7ae197b54373dac9 |
| Max. process+thread count | 705 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6444/4/console |
| versions | git=2.27.0 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 0s | Docker mode activated. | |
| -1 :x: | docker | 2m 10s | Docker failed to build run-specific yetus/hadoop:tp-25218}. |
| Subsystem | Report/Notes |
|---|---|
| GITHUB PR | https://github.com/apache/hadoop/pull/6444 |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6444/4/console |
| versions | git=2.34.1 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| -1 :x: | patch | 0m 55s | https://github.com/apache/hadoop/pull/6444 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help. |
| Subsystem | Report/Notes |
|---|---|
| GITHUB PR | https://github.com/apache/hadoop/pull/6444 |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6444/1/console |
| versions | git=2.44.0.windows.1 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.