hadoop
hadoop copied to clipboard
apache
HADOOP-18610. ABFS OAuth2 Token Provider support for Azure Workload Identity
Description of PR
Add support for Azure Active Directory (Azure AD) workload identities which integrate with the Kubernetes's native capabilities to federate with any external identity provider.
This PR is based on Haifeng Chen's patch attached to HADOOP-18610. I fixed a few typos and linter errors but did not modify the core functionality.
How was this patch tested?
New ABFS OAuth test configuration added for WorkloadIdentityTokenProvider. Complete test suite was run against Azure Blob Storage in Central US region.
:::: AGGREGATED TEST RESULT ::::
HNS-OAuth
[INFO] Results: [INFO] [ERROR] Failures: [ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider" [INFO] [ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 4 [INFO] Results: [INFO] [WARNING] Tests run: 587, Failures: 0, Errors: 0, Skipped: 99 [INFO] Results: [INFO] [ERROR] Errors: [ERROR] ITestAbfsTerasort.test_110_teragen:244->executeStage:206 » TestTimedOut test t... [INFO] [ERROR] Tests run: 339, Failures: 0, Errors: 1, Skipped: 56
HNS-SharedKey
[INFO] Results: [INFO] [ERROR] Failures: [ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider" [ERROR] Errors: [ERROR] TestExponentialRetryPolicy.testThrottlingIntercept:106 » KeyProvider Failure t... [INFO] [ERROR] Tests run: 141, Failures: 1, Errors: 1, Skipped: 5 [INFO] Results: [INFO] [WARNING] Tests run: 587, Failures: 0, Errors: 0, Skipped: 68 [INFO] Results: [INFO] [ERROR] Errors: [ERROR] ITestAbfsTerasort.test_110_teragen:244->executeStage:206 » TestTimedOut test t... [INFO] [ERROR] Tests run: 339, Failures: 0, Errors: 1, Skipped: 43
NonHNS-SharedKey
[INFO] Results: [INFO] [ERROR] Failures: [ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider" [ERROR] Errors: [ERROR] TestExponentialRetryPolicy.testThrottlingIntercept:106 » KeyProvider Failure t... [INFO] [ERROR] Tests run: 141, Failures: 1, Errors: 1, Skipped: 11 [INFO] Results: [INFO] [ERROR] Failures: [ERROR] ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS:181 Expecting org.apache.hadoop.security.AccessControlException with text "This request is not authorized to perform this operation using this permission.", 403 but got : "void" [INFO] [ERROR] Tests run: 587, Failures: 1, Errors: 0, Skipped: 277 [INFO] Results: [INFO] [ERROR] Errors: [ERROR] ITestAbfsTerasort.test_110_teragen:244->executeStage:206 » TestTimedOut test t... [INFO] [ERROR] Tests run: 339, Failures: 0, Errors: 1, Skipped: 46
AppendBlob-HNS-OAuth
[INFO] Results: [INFO] [ERROR] Failures: [ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider" [INFO] [ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 4 [INFO] Results: [INFO] [ERROR] Errors: [ERROR] ITestAzureBlobFileSystemLease.testTwoWritersCreateAppendNoInfiniteLease:177->twoWriters:165 » AbfsRestOperation [INFO] [ERROR] Tests run: 587, Failures: 0, Errors: 1, Skipped: 99 [INFO] Results: [INFO] [ERROR] Failures: [ERROR] ITestAbfsStreamStatistics.testAbfsStreamOps:140->Assert.assertTrue:42->Assert.fail:89 The actual value of 99 was not equal to the expected value [ERROR] Errors: [ERROR] ITestAbfsTerasort.test_110_teragen:244->executeStage:206 » TestTimedOut test t... [INFO] [ERROR] Tests run: 339, Failures: 1, Errors: 1, Skipped: 80
For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id
- [X] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
- [ ] If applicable, have you updated the
LICENSE,LICENSE-binary,NOTICE-binaryfiles?
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 1m 0s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 1s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
| +0 :ok: | markdownlint | 0m 1s | markdownlint was not available. | |
| +0 :ok: | yamllint | 0m 1s | yamllint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 45m 21s | trunk passed | |
| +1 :green_heart: | compile | 0m 42s | trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 | |
| +1 :green_heart: | compile | 0m 38s | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | |
| +1 :green_heart: | checkstyle | 0m 35s | trunk passed | |
| +1 :green_heart: | mvnsite | 0m 44s | trunk passed | |
| +1 :green_heart: | javadoc | 0m 44s | trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 37s | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | |
| +1 :green_heart: | spotbugs | 1m 10s | trunk passed | |
| +1 :green_heart: | shadedclient | 34m 9s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 0m 31s | the patch passed | |
| +1 :green_heart: | compile | 0m 31s | the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 | |
| +1 :green_heart: | javac | 0m 31s | the patch passed | |
| +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | |
| +1 :green_heart: | javac | 0m 28s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 0m 22s | /results-checkstyle-hadoop-tools_hadoop-azure.txt | hadoop-tools/hadoop-azure: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) |
| +1 :green_heart: | mvnsite | 0m 32s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 27s | the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 26s | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | |
| +1 :green_heart: | spotbugs | 1m 4s | the patch passed | |
| +1 :green_heart: | shadedclient | 34m 10s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 2m 10s | hadoop-azure in the patch passed. | |
| +1 :green_heart: | asflicense | 0m 40s | The patch does not generate ASF License warnings. | |
| 130m 46s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5953 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint yamllint |
| uname | Linux 44c8462208e4 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2632811a5e926fe54df0a2c367765f6dd3d83e18 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/1/testReport/ |
| Max. process+thread count | 699 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
@anmolanmol1234 - Thank you for the review. I added string constants.
Regarding tests, would you mind providing some guidance on what you are expecting? I added this documentation showing how the code changes can be tested using integration tests. I ran the integration tests and copied the output in the merge request description above.
If you're expecting unit tests instead of integration tests, then I would appreciate specific guidance on how that can be accomplished. This pull request includes changes to these existing classes:
- AzureADAuthenticator. No existing unit tests exist for this class and I don't see a simple way to mock the HTTP calls.
-
AbfsConfiguration. No existing unit tests exist to directly test
AbfsConfiguration.getTokenProvider(). I see that method called a few times indirectly as part of other tests though.
This pull request adds a new WorkloadTokenIdentityProvider class that extends AccessTokenProvider. Several other classes extend AccessTokenProvider, including:
- ClientCredsTokenProvider - Does not have unit tests.
- CustomTokenProviderAdapter - Has a single unit test with a comment implying E2E integration tests are preferred, but not possible in this case.
- MsiTokenProvider - Has an integration test class. I'm not sure how this is called when following the instructions for testing Azure.
- RefreshTokenBasedTokenProvider - Does not have unit tests.
- UserPasswordTokenProvider - Does not have unit tests.
Given the lack of unit tests in existing code, is it reasonable to conclude the new integration test documentation and test runs are sufficient in this case?
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 57s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 1s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
| +0 :ok: | markdownlint | 0m 1s | markdownlint was not available. | |
| +0 :ok: | yamllint | 0m 1s | yamllint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 45m 38s | trunk passed | |
| +1 :green_heart: | compile | 0m 42s | trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 0m 38s | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | |
| +1 :green_heart: | checkstyle | 0m 35s | trunk passed | |
| +1 :green_heart: | mvnsite | 0m 45s | trunk passed | |
| +1 :green_heart: | javadoc | 0m 43s | trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 38s | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | |
| +1 :green_heart: | spotbugs | 1m 10s | trunk passed | |
| +1 :green_heart: | shadedclient | 33m 49s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 0m 31s | the patch passed | |
| +1 :green_heart: | compile | 0m 32s | the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javac | 0m 32s | the patch passed | |
| +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | |
| +1 :green_heart: | javac | 0m 28s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 0m 22s | /results-checkstyle-hadoop-tools_hadoop-azure.txt | hadoop-tools/hadoop-azure: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) |
| +1 :green_heart: | mvnsite | 0m 30s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 28s | the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 26s | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | |
| +1 :green_heart: | spotbugs | 1m 3s | the patch passed | |
| +1 :green_heart: | shadedclient | 33m 52s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 2m 11s | hadoop-azure in the patch passed. | |
| +1 :green_heart: | asflicense | 0m 39s | The patch does not generate ASF License warnings. | |
| 130m 37s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5953 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint yamllint |
| uname | Linux 26f85c8852a9 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 088b5c39b20dc2910995463de764955c737d7898 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/2/testReport/ |
| Max. process+thread count | 672 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
@creste Is there any update on this PR? It is something I ideally need too and having similar pain points
@steven13cooper - I'm currently waiting on @anmolanmol1234 or another reviewer to approve the PR or at least provide guidance on how to add more tests.
Hi @steveloughran and @saxenapranav, could you please help review this PR? Thanks!
Lets add some tests around the change.
@saxenapranav - Please see my thoughts on tests here. It is not clear to me how to add tests to the code. Any guidance you could provide would be greatly appreciated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 53s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | |
| +0 :ok: | yamllint | 0m 0s | yamllint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 43m 30s | trunk passed | |
| +1 :green_heart: | compile | 0m 36s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 0m 34s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | checkstyle | 0m 30s | trunk passed | |
| +1 :green_heart: | mvnsite | 0m 40s | trunk passed | |
| +1 :green_heart: | javadoc | 0m 37s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 1m 2s | trunk passed | |
| +1 :green_heart: | shadedclient | 32m 21s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 0m 27s | the patch passed | |
| +1 :green_heart: | compile | 0m 29s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javac | 0m 29s | the patch passed | |
| +1 :green_heart: | compile | 0m 25s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | javac | 0m 25s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 0m 18s | the patch passed | |
| +1 :green_heart: | mvnsite | 0m 29s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 24s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 24s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 1m 2s | the patch passed | |
| +1 :green_heart: | shadedclient | 32m 16s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 2m 5s | hadoop-azure in the patch passed. | |
| +1 :green_heart: | asflicense | 0m 34s | The patch does not generate ASF License warnings. | |
| 123m 52s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5953 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint yamllint |
| uname | Linux b9a825722ed4 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d47fd7668d3515c1cb396b1da5711879f986f417 |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/3/testReport/ |
| Max. process+thread count | 556 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
Lets add some tests around the change.
@saxenapranav - Please see my thoughts on tests here. It is not clear to me how to add tests to the code. Any guidance you could provide would be greatly appreciated.
Thanks for taking the comments. Have added my thought-process for the test.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 24s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 1s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
| +0 :ok: | markdownlint | 0m 1s | markdownlint was not available. | |
| +0 :ok: | yamllint | 0m 1s | yamllint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 32m 15s | trunk passed | |
| +1 :green_heart: | compile | 0m 22s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 0m 21s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | checkstyle | 0m 20s | trunk passed | |
| +1 :green_heart: | mvnsite | 0m 25s | trunk passed | |
| +1 :green_heart: | javadoc | 0m 23s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 21s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 0m 43s | trunk passed | |
| +1 :green_heart: | shadedclient | 19m 30s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 0m 17s | the patch passed | |
| +1 :green_heart: | compile | 0m 19s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javac | 0m 19s | the patch passed | |
| +1 :green_heart: | compile | 0m 17s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | javac | 0m 17s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 0m 10s | the patch passed | |
| +1 :green_heart: | mvnsite | 0m 19s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 17s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 18s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 0m 43s | the patch passed | |
| +1 :green_heart: | shadedclient | 19m 25s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 1m 43s | hadoop-azure in the patch passed. | |
| +1 :green_heart: | asflicense | 0m 23s | The patch does not generate ASF License warnings. | |
| 81m 30s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5953 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint yamllint |
| uname | Linux e0b2e5a7f124 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 524ee187da948b699985abf276782a026b95db95 |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/5/testReport/ |
| Max. process+thread count | 555 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 34s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | |
| +0 :ok: | yamllint | 0m 0s | yamllint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 42m 58s | trunk passed | |
| +1 :green_heart: | compile | 0m 33s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 0m 34s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | checkstyle | 0m 31s | trunk passed | |
| +1 :green_heart: | mvnsite | 0m 40s | trunk passed | |
| +1 :green_heart: | javadoc | 0m 37s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 1m 3s | trunk passed | |
| +1 :green_heart: | shadedclient | 32m 29s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 0m 26s | the patch passed | |
| +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javac | 0m 28s | the patch passed | |
| +1 :green_heart: | compile | 0m 25s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | javac | 0m 25s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 0m 20s | the patch passed | |
| +1 :green_heart: | mvnsite | 0m 28s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 26s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 24s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 1m 2s | the patch passed | |
| +1 :green_heart: | shadedclient | 32m 19s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 1m 58s | hadoop-azure in the patch passed. | |
| +1 :green_heart: | asflicense | 0m 34s | The patch does not generate ASF License warnings. | |
| 123m 3s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5953 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint yamllint |
| uname | Linux eb709fc94eda 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7d4cdb33d11df582753b1bd4e665809d2f9c53bb |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/4/testReport/ |
| Max. process+thread count | 561 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
@saxenapranav - Thank you for the guidance. I added unit tests and addressed all other feedback.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 53s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | |
| +0 :ok: | yamllint | 0m 0s | yamllint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 41m 27s | trunk passed | |
| +1 :green_heart: | compile | 0m 37s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 0m 34s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | checkstyle | 0m 29s | trunk passed | |
| +1 :green_heart: | mvnsite | 0m 39s | trunk passed | |
| +1 :green_heart: | javadoc | 0m 36s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 1m 4s | trunk passed | |
| +1 :green_heart: | shadedclient | 32m 18s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 0m 27s | the patch passed | |
| +1 :green_heart: | compile | 0m 29s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javac | 0m 29s | the patch passed | |
| +1 :green_heart: | compile | 0m 25s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | javac | 0m 25s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 0m 18s | /results-checkstyle-hadoop-tools_hadoop-azure.txt | hadoop-tools/hadoop-azure: The patch generated 2 new + 2 unchanged - 0 fixed = 4 total (was 2) |
| +1 :green_heart: | mvnsite | 0m 28s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 25s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 25s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 1m 2s | the patch passed | |
| +1 :green_heart: | shadedclient | 32m 34s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 2m 7s | hadoop-azure in the patch passed. | |
| +1 :green_heart: | asflicense | 0m 34s | The patch does not generate ASF License warnings. | |
| 122m 4s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5953 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint yamllint |
| uname | Linux 9de056802853 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 08da2eaf7f038d1df0948e96b6dbe7e8bda92282 |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/6/testReport/ |
| Max. process+thread count | 633 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 55s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | |
| +0 :ok: | yamllint | 0m 0s | yamllint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 41m 46s | trunk passed | |
| +1 :green_heart: | compile | 0m 36s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 0m 34s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | checkstyle | 0m 31s | trunk passed | |
| +1 :green_heart: | mvnsite | 0m 39s | trunk passed | |
| +1 :green_heart: | javadoc | 0m 36s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 1m 2s | trunk passed | |
| +1 :green_heart: | shadedclient | 32m 19s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| -1 :x: | mvninstall | 0m 21s | /patch-mvninstall-hadoop-tools_hadoop-azure.txt | hadoop-azure in the patch failed. |
| -1 :x: | compile | 0m 28s | /patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt | hadoop-azure in the patch failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | javac | 0m 28s | /patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt | hadoop-azure in the patch failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | compile | 0m 22s | /patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt | hadoop-azure in the patch failed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08. |
| -1 :x: | javac | 0m 22s | /patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt | hadoop-azure in the patch failed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08. |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 0m 18s | the patch passed | |
| -1 :x: | mvnsite | 0m 23s | /patch-mvnsite-hadoop-tools_hadoop-azure.txt | hadoop-azure in the patch failed. |
| +1 :green_heart: | javadoc | 0m 25s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 23s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| -1 :x: | spotbugs | 0m 23s | /patch-spotbugs-hadoop-tools_hadoop-azure.txt | hadoop-azure in the patch failed. |
| +1 :green_heart: | shadedclient | 34m 25s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 0m 26s | /patch-unit-hadoop-tools_hadoop-azure.txt | hadoop-azure in the patch failed. |
| +1 :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | |
| 119m 17s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/7/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5953 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint yamllint |
| uname | Linux 55b90fe0f2f6 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / fe75cd3b46e78fc49339c2354f3101e3369d2342 |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/7/testReport/ |
| Max. process+thread count | 561 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | |
| +0 :ok: | yamllint | 0m 0s | yamllint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 41m 56s | trunk passed | |
| +1 :green_heart: | compile | 0m 36s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 0m 32s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | checkstyle | 0m 30s | trunk passed | |
| +1 :green_heart: | mvnsite | 0m 37s | trunk passed | |
| +1 :green_heart: | javadoc | 0m 37s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 1m 4s | trunk passed | |
| +1 :green_heart: | shadedclient | 32m 36s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| -1 :x: | mvninstall | 0m 22s | /patch-mvninstall-hadoop-tools_hadoop-azure.txt | hadoop-azure in the patch failed. |
| -1 :x: | compile | 0m 28s | /patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt | hadoop-azure in the patch failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | javac | 0m 28s | /patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt | hadoop-azure in the patch failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | compile | 0m 22s | /patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt | hadoop-azure in the patch failed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08. |
| -1 :x: | javac | 0m 22s | /patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt | hadoop-azure in the patch failed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08. |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 0m 18s | the patch passed | |
| -1 :x: | mvnsite | 0m 24s | /patch-mvnsite-hadoop-tools_hadoop-azure.txt | hadoop-azure in the patch failed. |
| +1 :green_heart: | javadoc | 0m 24s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 24s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| -1 :x: | spotbugs | 0m 23s | /patch-spotbugs-hadoop-tools_hadoop-azure.txt | hadoop-azure in the patch failed. |
| +1 :green_heart: | shadedclient | 34m 39s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 0m 27s | /patch-unit-hadoop-tools_hadoop-azure.txt | hadoop-azure in the patch failed. |
| +1 :green_heart: | asflicense | 0m 34s | The patch does not generate ASF License warnings. | |
| 119m 32s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/8/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5953 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint yamllint |
| uname | Linux 2db524880db4 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / fe75cd3b46e78fc49339c2354f3101e3369d2342 |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/8/testReport/ |
| Max. process+thread count | 561 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/8/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 53s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 1s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
| +0 :ok: | markdownlint | 0m 1s | markdownlint was not available. | |
| +0 :ok: | yamllint | 0m 1s | yamllint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 43m 7s | trunk passed | |
| +1 :green_heart: | compile | 0m 37s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 0m 34s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | checkstyle | 0m 30s | trunk passed | |
| +1 :green_heart: | mvnsite | 0m 38s | trunk passed | |
| +1 :green_heart: | javadoc | 0m 37s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 1m 3s | trunk passed | |
| +1 :green_heart: | shadedclient | 32m 27s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 0m 28s | the patch passed | |
| +1 :green_heart: | compile | 0m 29s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javac | 0m 29s | the patch passed | |
| +1 :green_heart: | compile | 0m 26s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | javac | 0m 26s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 0m 19s | the patch passed | |
| +1 :green_heart: | mvnsite | 0m 28s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 26s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 25s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 1m 3s | the patch passed | |
| +1 :green_heart: | shadedclient | 32m 21s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 2m 4s | hadoop-azure in the patch passed. | |
| +1 :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | |
| 123m 45s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/9/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5953 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint yamllint |
| uname | Linux 14669132ef6f 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 81c1b62e4ca3bdef13d6e826307604bac65f91d4 |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/9/testReport/ |
| Max. process+thread count | 560 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/9/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 19m 56s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | |
| +0 :ok: | yamllint | 0m 0s | yamllint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 50m 40s | trunk passed | |
| +1 :green_heart: | compile | 0m 41s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 0m 36s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | checkstyle | 0m 30s | trunk passed | |
| +1 :green_heart: | mvnsite | 0m 40s | trunk passed | |
| +1 :green_heart: | javadoc | 0m 38s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 31s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 1m 10s | trunk passed | |
| -1 :x: | shadedclient | 5m 28s | branch has errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| -1 :x: | mvninstall | 0m 22s | /patch-mvninstall-hadoop-tools_hadoop-azure.txt | hadoop-azure in the patch failed. |
| -1 :x: | compile | 0m 23s | /patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt | hadoop-azure in the patch failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | javac | 0m 23s | /patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt | hadoop-azure in the patch failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | compile | 0m 23s | /patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt | hadoop-azure in the patch failed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08. |
| -1 :x: | javac | 0m 23s | /patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt | hadoop-azure in the patch failed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08. |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 0m 21s | /buildtool-patch-checkstyle-hadoop-tools_hadoop-azure.txt | The patch fails to run checkstyle in hadoop-azure |
| -1 :x: | mvnsite | 0m 22s | /patch-mvnsite-hadoop-tools_hadoop-azure.txt | hadoop-azure in the patch failed. |
| -1 :x: | javadoc | 0m 23s | /patch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt | hadoop-azure in the patch failed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | javadoc | 0m 23s | /patch-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt | hadoop-azure in the patch failed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08. |
| -1 :x: | spotbugs | 0m 23s | /patch-spotbugs-hadoop-tools_hadoop-azure.txt | hadoop-azure in the patch failed. |
| +1 :green_heart: | shadedclient | 4m 32s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 0m 23s | /patch-unit-hadoop-tools_hadoop-azure.txt | hadoop-azure in the patch failed. |
| +0 :ok: | asflicense | 0m 23s | ASF License check generated no output? | |
| 90m 54s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/10/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5953 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint yamllint |
| uname | Linux bfccdeee4eba 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 3aeacfa71fa49d2cf1d34f44a82881810955186a |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/10/testReport/ |
| Max. process+thread count | 88 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/10/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
This behaviour of the AAD workload identity is not well documented, but the AAD workload identity webhook injects the following env variables into pods.
https://azure.github.io/azure-workload-identity/docs/quick-start.html#7-deploy-workload
| Environment variable | Description |
|---|---|
AZURE_AUTHORITY_HOST |
The Azure Active Directory (AAD) endpoint. |
AZURE_CLIENT_ID |
The client ID of the AAD application or user-assigned managed identity. |
AZURE_TENANT_ID |
The tenant ID of the registered AAD application or user-assigned managed identity. |
AZURE_FEDERATED_TOKEN_FILE |
The path of the projected service account token file. |
WorkloadIdentityCredential provided by Azure SDK also reads these env variables by default.
- https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/identity/azure-identity/azure/identity/_credentials/workload_identity.py#L70-L72
- https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/azidentity/workload_identity.go#L64-L80
I think it is better to make the following config properties optional and use values provided by the env variables above by default.
-
fs.azure.account.oauth2.msi.authority -
fs.azure.account.oauth2.client.id -
fs.azure.account.oauth2.msi.tenant -
fs.azure.account.oauth2.token.file
These values provided by Hadoop config are redundant in general and potentially cause inconsistency with the correct properties of federated identities.
you can use env var resolution within a hadoop core-site file; which lets you at the values with defaults when unset. on locked down config loading (oozie etc) then only the default is valid.
${env.LOCAL_DIRS:-some.default}/
so: no need to add explicit resolution, just document or set as default. example, s3a uses temp dirs in yarn containers automatically.
<property>
<name>fs.s3a.buffer.dir</name>
<value>${env.LOCAL_DIRS:-${hadoop.tmp.dir}}/s3a</value>
<description>Comma separated list of directories that will be used to buffer file
uploads to.
Yarn container path will be used as default value on yarn applications,
otherwise fall back to hadoop.tmp.dir
</description>
</property>
@sugibuchi @steveloughran - I added the Azure environment variables to the hadoop config documented in the readme. Please take a look when you get a chance.
@steveloughran I agree. The env ver resolution looks the best solution for this.
@creste Thank you very much for this prompt update. About the descriptions of the four properties, I think we can simply copy-paste the descriptions provided by ADD Workload identity documentation.
-
fs.azure.account.oauth2.msi.tenant: The tenant ID of the registered AAD application or user-assigned managed identity. -
fs.azure.account.oauth2.client.id: The client ID of the AAD application or user-assigned managed identity. -
fs.azure.account.oauth2.token.file: The path of the projected service account token file.
About the description of the auth method:
OAuth 2.0 tokens are written to a file that is only accessible from the executing pod (
/var/run/secrets/azure/tokens/azure-identity-token). The issued credentials can be used to authenticate.
This is not precise. The token files injected by the AAD workload identity webhook are files of "projected service account tokens" issued by Kubernetes clusters. They are not OAuth2 access tokens for accessing Azure resources.
https://azure.github.io/azure-workload-identity/docs/introduction.html#how-it-works
I propose to update the description of this auth method like:
With a projected service account token injected by the Azure Workload Identity webhook, make a request of the Azure Active Directry endpoint to retrieve access tokens. The required properties for this authentication method are automatically injected into the executing pod as environment variables by the AAD Workload Identity webhook.
@sugibuchi - Thank you for the additional comments.
About the descriptions of the four properties, I think we can simply copy-paste the descriptions provided by ADD Workload identity documentation.
- fs.azure.account.oauth2.msi.tenant: The tenant ID of the registered AAD application or user-assigned managed identity.
- fs.azure.account.oauth2.client.id: The client ID of the AAD application or user-assigned managed identity.
- fs.azure.account.oauth2.token.file: The path of the projected service account token file.
The current descriptions of the properties were copied from other parts of the README. For example, see the property descriptions for MSITokenProvider. @steveloughran or @anmolanmol1234 - what descriptions should the README use for those properties?
About the description of the auth method:
OAuth 2.0 tokens are written to a file that is only accessible from the executing pod (
/var/run/secrets/azure/tokens/azure-identity-token). The issued credentials can be used to authenticate.This is not precise. The token files injected by the AAD workload identity webhook are files of "projected service account tokens" issued by Kubernetes clusters. They are not OAuth2 access tokens for accessing Azure resources.
I propose to update the description of this auth method like:
With a projected service account token injected by the Azure Workload Identity webhook, make a request of the Azure Active Directry endpoint to retrieve access tokens. The required properties for this authentication method are automatically injected into the executing pod as environment variables by the AAD Workload Identity webhook.
I have no preference, but since this text was also based on other descriptions in the README I would appreciate input from a maintainer before making the change. @steveloughran or @anmolanmol1234 - any thoughts on this?
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 56s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | |
| +0 :ok: | yamllint | 0m 0s | yamllint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 41m 37s | trunk passed | |
| +1 :green_heart: | compile | 0m 35s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 0m 34s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | checkstyle | 0m 31s | trunk passed | |
| +1 :green_heart: | mvnsite | 0m 38s | trunk passed | |
| +1 :green_heart: | javadoc | 0m 36s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 1m 3s | trunk passed | |
| +1 :green_heart: | shadedclient | 32m 19s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 0m 29s | the patch passed | |
| +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javac | 0m 28s | the patch passed | |
| +1 :green_heart: | compile | 0m 26s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | javac | 0m 26s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 0m 20s | /results-checkstyle-hadoop-tools_hadoop-azure.txt | hadoop-tools/hadoop-azure: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) |
| +1 :green_heart: | mvnsite | 0m 28s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 25s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 24s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 1m 2s | the patch passed | |
| +1 :green_heart: | shadedclient | 32m 46s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 2m 5s | hadoop-azure in the patch passed. | |
| +1 :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | |
| 122m 34s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/11/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5953 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint yamllint |
| uname | Linux 21bc237e9b41 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 329e3c900bc2668651aa812fa075501c494652df |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/11/testReport/ |
| Max. process+thread count | 725 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/11/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 58s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | |
| +0 :ok: | yamllint | 0m 0s | yamllint was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 42m 12s | trunk passed | |
| +1 :green_heart: | compile | 0m 36s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 0m 33s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | checkstyle | 0m 30s | trunk passed | |
| +1 :green_heart: | mvnsite | 0m 39s | trunk passed | |
| +1 :green_heart: | javadoc | 0m 35s | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 32s | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 1m 1s | trunk passed | |
| +1 :green_heart: | shadedclient | 32m 23s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 0m 28s | the patch passed | |
| +1 :green_heart: | compile | 0m 29s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javac | 0m 29s | the patch passed | |
| +1 :green_heart: | compile | 0m 25s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | javac | 0m 25s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 0m 19s | the patch passed | |
| +1 :green_heart: | mvnsite | 0m 30s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 25s | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 0m 25s | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 | |
| +1 :green_heart: | spotbugs | 1m 3s | the patch passed | |
| +1 :green_heart: | shadedclient | 32m 15s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 2m 5s | hadoop-azure in the patch passed. | |
| +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | |
| 122m 43s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/12/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5953 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint yamllint |
| uname | Linux 88c4d7719585 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f8be1ca268e59949d860804a3ac1ca6180fca1de |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/12/testReport/ |
| Max. process+thread count | 552 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5953/12/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
Hi @creste , there is a need for this change to merge to trunk. Can you please share the plan on completing this PR.
Please let me know if I can help you anyway to address the review comments to get this change in. Thanks.
@snvijaya - My team is no longer using Hadoop and has moved on to another project so I am unable to commit to completing this PR. Feel free to address the feedback and make any changes needed to get this merged.
Thanks @creste . @anujmodi2021 Will pick this up. As the PR is raised from a forked repo, we will not be able to make changes to the same dev branch from which the PR is raised. Will re-raise the PR and address the comments.