hadoop
hadoop copied to clipboard
YARN-5871. [RESERVATION] Add support for reservation-based routing.
YARN-5871. [RESERVATION] Add support for reservation-based routing.
When completing the pr of YARN-11177, I found that YARN Federation has limited support for Reservation System.
After looking at Jira, I found that YARN-5871 implements some of the functions of Router Reservation System, I continue to follow this pr. However, the original code of YARN-5871 has changed too much for the Router Policy And it can no longer be directly merged into the current version of trunk. I refer to this part of the code to try to be compatible with the current version of the Router.
After reading the code of this PR, I will describe the design idea of this PR. This part of the work is in progress, first ensure that the PR can be compiled normally and passed the unit test.
:broken_heart: -1 overall
Vote | Subsystem | Runtime | Logfile | Comment |
---|---|---|---|---|
+0 :ok: | reexec | 1m 11s | Docker mode activated. | |
_ Prechecks _ | ||||
+1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
+0 :ok: | codespell | 0m 1s | codespell was not available. | |
+0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
+0 :ok: | buf | 0m 1s | buf was not available. | |
+0 :ok: | buf | 0m 1s | buf was not available. | |
+1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
+1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | |
_ trunk Compile Tests _ | ||||
+0 :ok: | mvndep | 15m 8s | Maven dependency ordering for branch | |
+1 :green_heart: | mvninstall | 29m 49s | trunk passed | |
+1 :green_heart: | compile | 4m 36s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | compile | 3m 46s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | checkstyle | 1m 37s | trunk passed | |
+1 :green_heart: | mvnsite | 2m 12s | trunk passed | |
+1 :green_heart: | javadoc | 2m 2s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | javadoc | 1m 43s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | spotbugs | 4m 17s | trunk passed | |
+1 :green_heart: | shadedclient | 25m 19s | branch has no errors when building and testing our client artifacts. | |
_ Patch Compile Tests _ | ||||
+0 :ok: | mvndep | 0m 28s | Maven dependency ordering for patch | |
-1 :x: | mvninstall | 0m 34s | /patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt | hadoop-yarn-server-common in the patch failed. |
-1 :x: | mvninstall | 0m 35s | /patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt | hadoop-yarn-server-resourcemanager in the patch failed. |
-1 :x: | compile | 0m 44s | /patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt | hadoop-yarn-server in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
-1 :x: | cc | 0m 44s | /patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt | hadoop-yarn-server in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
-1 :x: | javac | 0m 44s | /patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt | hadoop-yarn-server in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
-1 :x: | compile | 0m 39s | /patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt | hadoop-yarn-server in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07. |
-1 :x: | cc | 0m 39s | /patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt | hadoop-yarn-server in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07. |
-1 :x: | javac | 0m 39s | /patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt | hadoop-yarn-server in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07. |
+1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
-0 :warning: | checkstyle | 1m 14s | /results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 44 new + 13 unchanged - 0 fixed = 57 total (was 13) |
-1 :x: | mvnsite | 0m 39s | /patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt | hadoop-yarn-server-common in the patch failed. |
-1 :x: | mvnsite | 0m 37s | /patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt | hadoop-yarn-server-resourcemanager in the patch failed. |
-1 :x: | javadoc | 0m 33s | /patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt | hadoop-yarn-server-resourcemanager in the patch failed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
-1 :x: | javadoc | 0m 44s | /results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 20 new + 343 unchanged - 0 fixed = 363 total (was 343) |
-1 :x: | spotbugs | 0m 36s | /patch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt | hadoop-yarn-server-common in the patch failed. |
-1 :x: | spotbugs | 0m 38s | /patch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt | hadoop-yarn-server-resourcemanager in the patch failed. |
-1 :x: | shadedclient | 17m 22s | patch has errors when building and testing our client artifacts. | |
_ Other Tests _ | ||||
-1 :x: | unit | 0m 40s | /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt | hadoop-yarn-server-common in the patch failed. |
-1 :x: | unit | 0m 39s | /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt | hadoop-yarn-server-resourcemanager in the patch failed. |
+1 :green_heart: | asflicense | 0m 39s | The patch does not generate ASF License warnings. | |
112m 41s |
Subsystem | Report/Notes |
---|---|
Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/2/artifact/out/Dockerfile |
GITHUB PR | https://github.com/apache/hadoop/pull/4632 |
Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat |
uname | Linux 7b0e6b168531 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | dev-support/bin/hadoop.sh |
git revision | trunk / 9f72ed7a67cc64766abf55f17db3686149fe2751 |
Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/2/testReport/ |
Max. process+thread count | 529 (vs. ulimit of 5500) |
modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server |
Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/2/console |
versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
Vote | Subsystem | Runtime | Logfile | Comment |
---|---|---|---|---|
+0 :ok: | reexec | 1m 15s | Docker mode activated. | |
_ Prechecks _ | ||||
+1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
+0 :ok: | codespell | 0m 1s | codespell was not available. | |
+0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
+0 :ok: | buf | 0m 1s | buf was not available. | |
+0 :ok: | buf | 0m 1s | buf was not available. | |
+1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
-1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | |
_ trunk Compile Tests _ | ||||
+0 :ok: | mvndep | 15m 1s | Maven dependency ordering for branch | |
+1 :green_heart: | mvninstall | 28m 34s | trunk passed | |
+1 :green_heart: | compile | 4m 17s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | compile | 3m 34s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | checkstyle | 1m 35s | trunk passed | |
+1 :green_heart: | mvnsite | 2m 11s | trunk passed | |
+1 :green_heart: | javadoc | 1m 59s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | javadoc | 1m 43s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | spotbugs | 4m 20s | trunk passed | |
+1 :green_heart: | shadedclient | 25m 20s | branch has no errors when building and testing our client artifacts. | |
_ Patch Compile Tests _ | ||||
+0 :ok: | mvndep | 0m 32s | Maven dependency ordering for patch | |
+1 :green_heart: | mvninstall | 1m 42s | the patch passed | |
+1 :green_heart: | compile | 4m 29s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | cc | 4m 29s | the patch passed | |
-1 :x: | javac | 4m 29s | /results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 3 new + 447 unchanged - 0 fixed = 450 total (was 447) |
+1 :green_heart: | compile | 3m 50s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | cc | 3m 50s | the patch passed | |
-1 :x: | javac | 3m 50s | /results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new + 371 unchanged - 0 fixed = 374 total (was 371) |
+1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
-0 :warning: | checkstyle | 1m 20s | /results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 44 new + 13 unchanged - 0 fixed = 57 total (was 13) |
+1 :green_heart: | mvnsite | 1m 54s | the patch passed | |
+1 :green_heart: | javadoc | 1m 27s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | javadoc | 1m 18s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | spotbugs | 3m 46s | the patch passed | |
+1 :green_heart: | shadedclient | 25m 24s | patch has no errors when building and testing our client artifacts. | |
_ Other Tests _ | ||||
-1 :x: | unit | 3m 7s | /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt | hadoop-yarn-server-common in the patch passed. |
+1 :green_heart: | unit | 117m 39s | hadoop-yarn-server-resourcemanager in the patch passed. | |
+1 :green_heart: | asflicense | 0m 44s | The patch does not generate ASF License warnings. | |
256m 45s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.yarn.server.federation.policies.TestRouterPolicyFacade |
Subsystem | Report/Notes |
---|---|
Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/1/artifact/out/Dockerfile |
GITHUB PR | https://github.com/apache/hadoop/pull/4632 |
Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat |
uname | Linux 985dd3feaa1f 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | dev-support/bin/hadoop.sh |
git revision | trunk / 73940608985def6b703b13ba2f35cf47e1351601 |
Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/1/testReport/ |
Max. process+thread count | 903 (vs. ulimit of 5500) |
modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server |
Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/1/console |
versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
Vote | Subsystem | Runtime | Logfile | Comment |
---|---|---|---|---|
+0 :ok: | reexec | 0m 38s | Docker mode activated. | |
_ Prechecks _ | ||||
+1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | |
+0 :ok: | codespell | 0m 0s | codespell was not available. | |
+0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
+0 :ok: | buf | 0m 0s | buf was not available. | |
+0 :ok: | buf | 0m 0s | buf was not available. | |
+1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
+1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | |
_ trunk Compile Tests _ | ||||
+0 :ok: | mvndep | 15m 15s | Maven dependency ordering for branch | |
+1 :green_heart: | mvninstall | 25m 26s | trunk passed | |
+1 :green_heart: | compile | 4m 4s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | compile | 3m 30s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | checkstyle | 1m 32s | trunk passed | |
+1 :green_heart: | mvnsite | 2m 20s | trunk passed | |
+1 :green_heart: | javadoc | 2m 6s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | javadoc | 1m 55s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | spotbugs | 3m 54s | trunk passed | |
+1 :green_heart: | shadedclient | 21m 37s | branch has no errors when building and testing our client artifacts. | |
_ Patch Compile Tests _ | ||||
+0 :ok: | mvndep | 0m 31s | Maven dependency ordering for patch | |
+1 :green_heart: | mvninstall | 1m 40s | the patch passed | |
+1 :green_heart: | compile | 3m 49s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | cc | 3m 49s | the patch passed | |
-1 :x: | javac | 3m 49s | /results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 3 new + 444 unchanged - 0 fixed = 447 total (was 444) |
+1 :green_heart: | compile | 3m 17s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | cc | 3m 17s | the patch passed | |
-1 :x: | javac | 3m 17s | /results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new + 368 unchanged - 0 fixed = 371 total (was 368) |
+1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
+1 :green_heart: | checkstyle | 1m 13s | the patch passed | |
+1 :green_heart: | mvnsite | 1m 50s | the patch passed | |
+1 :green_heart: | javadoc | 1m 32s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | javadoc | 1m 27s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | spotbugs | 3m 37s | the patch passed | |
+1 :green_heart: | shadedclient | 22m 54s | patch has no errors when building and testing our client artifacts. | |
_ Other Tests _ | ||||
-1 :x: | unit | 3m 7s | /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt | hadoop-yarn-server-common in the patch passed. |
+1 :green_heart: | unit | 98m 52s | hadoop-yarn-server-resourcemanager in the patch passed. | |
+1 :green_heart: | asflicense | 0m 51s | The patch does not generate ASF License warnings. | |
227m 33s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.yarn.server.federation.policies.TestRouterPolicyFacade |
Subsystem | Report/Notes |
---|---|
Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/3/artifact/out/Dockerfile |
GITHUB PR | https://github.com/apache/hadoop/pull/4632 |
Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat |
uname | Linux 53685874c809 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | dev-support/bin/hadoop.sh |
git revision | trunk / d15a2edefb918b4f573541c9c81372922ef93b89 |
Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/3/testReport/ |
Max. process+thread count | 1000 (vs. ulimit of 5500) |
modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server |
Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/3/console |
versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
@goiri I have integrated the code of YARN-5871, tested it locally, and it runs normally. There are 3 proto compilation prompts in javac, which I personally think can be ignored. I am sorting out the design documents, and it is expected to be available in 1-2 days. Please help to review code, thank you very much!
I don't plan to split into multiple prs, because a lot of code is related to proto, I will explain this part in the documentation.
:broken_heart: -1 overall
Vote | Subsystem | Runtime | Logfile | Comment |
---|---|---|---|---|
+0 :ok: | reexec | 0m 51s | Docker mode activated. | |
_ Prechecks _ | ||||
+1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | |
+0 :ok: | codespell | 0m 1s | codespell was not available. | |
+0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
+0 :ok: | buf | 0m 1s | buf was not available. | |
+0 :ok: | buf | 0m 1s | buf was not available. | |
+1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
+1 :green_heart: | test4tests | 0m 0s | The patch appears to include 10 new or modified test files. | |
_ trunk Compile Tests _ | ||||
+0 :ok: | mvndep | 15m 9s | Maven dependency ordering for branch | |
+1 :green_heart: | mvninstall | 25m 21s | trunk passed | |
+1 :green_heart: | compile | 4m 3s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | compile | 3m 28s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | checkstyle | 1m 30s | trunk passed | |
+1 :green_heart: | mvnsite | 2m 18s | trunk passed | |
+1 :green_heart: | javadoc | 2m 8s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | javadoc | 1m 54s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | spotbugs | 3m 53s | trunk passed | |
+1 :green_heart: | shadedclient | 21m 43s | branch has no errors when building and testing our client artifacts. | |
_ Patch Compile Tests _ | ||||
+0 :ok: | mvndep | 0m 31s | Maven dependency ordering for patch | |
+1 :green_heart: | mvninstall | 1m 40s | the patch passed | |
+1 :green_heart: | compile | 3m 47s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | cc | 3m 47s | the patch passed | |
-1 :x: | javac | 3m 47s | /results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 3 new + 444 unchanged - 0 fixed = 447 total (was 444) |
+1 :green_heart: | compile | 3m 17s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | cc | 3m 17s | the patch passed | |
-1 :x: | javac | 3m 17s | /results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new + 368 unchanged - 0 fixed = 371 total (was 368) |
+1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
-0 :warning: | checkstyle | 1m 14s | /results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 16 new + 13 unchanged - 0 fixed = 29 total (was 13) |
+1 :green_heart: | mvnsite | 1m 49s | the patch passed | |
+1 :green_heart: | javadoc | 1m 29s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | javadoc | 1m 26s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | spotbugs | 3m 40s | the patch passed | |
+1 :green_heart: | shadedclient | 21m 15s | patch has no errors when building and testing our client artifacts. | |
_ Other Tests _ | ||||
+1 :green_heart: | unit | 3m 2s | hadoop-yarn-server-common in the patch passed. | |
+1 :green_heart: | unit | 99m 34s | hadoop-yarn-server-resourcemanager in the patch passed. | |
+1 :green_heart: | asflicense | 0m 48s | The patch does not generate ASF License warnings. | |
226m 26s |
Subsystem | Report/Notes |
---|---|
Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/4/artifact/out/Dockerfile |
GITHUB PR | https://github.com/apache/hadoop/pull/4632 |
Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat |
uname | Linux 97f1041c6b6c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | dev-support/bin/hadoop.sh |
git revision | trunk / 8bccea9836e75a092ae904a1063bd7c5ddcfbdf9 |
Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/4/testReport/ |
Max. process+thread count | 980 (vs. ulimit of 5500) |
modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server |
Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/4/console |
versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
Vote | Subsystem | Runtime | Logfile | Comment |
---|---|---|---|---|
+0 :ok: | reexec | 0m 58s | Docker mode activated. | |
_ Prechecks _ | ||||
+1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | |
+0 :ok: | codespell | 0m 1s | codespell was not available. | |
+0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
+0 :ok: | buf | 0m 1s | buf was not available. | |
+0 :ok: | buf | 0m 1s | buf was not available. | |
+1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
+1 :green_heart: | test4tests | 0m 0s | The patch appears to include 10 new or modified test files. | |
_ trunk Compile Tests _ | ||||
+0 :ok: | mvndep | 14m 48s | Maven dependency ordering for branch | |
+1 :green_heart: | mvninstall | 28m 21s | trunk passed | |
+1 :green_heart: | compile | 4m 13s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | compile | 3m 25s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | checkstyle | 1m 30s | trunk passed | |
+1 :green_heart: | mvnsite | 2m 2s | trunk passed | |
+1 :green_heart: | javadoc | 1m 50s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | javadoc | 1m 35s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | spotbugs | 3m 48s | trunk passed | |
+1 :green_heart: | shadedclient | 25m 53s | branch has no errors when building and testing our client artifacts. | |
_ Patch Compile Tests _ | ||||
+0 :ok: | mvndep | 0m 30s | Maven dependency ordering for patch | |
+1 :green_heart: | mvninstall | 1m 46s | the patch passed | |
+1 :green_heart: | compile | 4m 48s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | cc | 4m 48s | the patch passed | |
-1 :x: | javac | 4m 48s | /results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 3 new + 444 unchanged - 0 fixed = 447 total (was 444) |
+1 :green_heart: | compile | 3m 31s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | cc | 3m 31s | the patch passed | |
-1 :x: | javac | 3m 31s | /results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new + 368 unchanged - 0 fixed = 371 total (was 368) |
+1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
-0 :warning: | checkstyle | 1m 13s | /results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 17 new + 13 unchanged - 0 fixed = 30 total (was 13) |
+1 :green_heart: | mvnsite | 1m 41s | the patch passed | |
+1 :green_heart: | javadoc | 1m 23s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | javadoc | 1m 16s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | spotbugs | 3m 39s | the patch passed | |
+1 :green_heart: | shadedclient | 24m 30s | patch has no errors when building and testing our client artifacts. | |
_ Other Tests _ | ||||
+1 :green_heart: | unit | 2m 48s | hadoop-yarn-server-common in the patch passed. | |
+1 :green_heart: | unit | 102m 53s | hadoop-yarn-server-resourcemanager in the patch passed. | |
+1 :green_heart: | asflicense | 0m 42s | The patch does not generate ASF License warnings. | |
239m 17s |
Subsystem | Report/Notes |
---|---|
Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/5/artifact/out/Dockerfile |
GITHUB PR | https://github.com/apache/hadoop/pull/4632 |
Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat |
uname | Linux c6d52ff37c81 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | dev-support/bin/hadoop.sh |
git revision | trunk / e79188b6cd58b7a1f4bfe9e768c0f0d373aed5d2 |
Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/5/testReport/ |
Max. process+thread count | 913 (vs. ulimit of 5500) |
modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server |
Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/5/console |
versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
Vote | Subsystem | Runtime | Logfile | Comment |
---|---|---|---|---|
+0 :ok: | reexec | 1m 10s | Docker mode activated. | |
_ Prechecks _ | ||||
+1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | |
+0 :ok: | codespell | 0m 1s | codespell was not available. | |
+0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
+0 :ok: | buf | 0m 1s | buf was not available. | |
+0 :ok: | buf | 0m 1s | buf was not available. | |
+1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
+1 :green_heart: | test4tests | 0m 0s | The patch appears to include 10 new or modified test files. | |
_ trunk Compile Tests _ | ||||
+0 :ok: | mvndep | 14m 59s | Maven dependency ordering for branch | |
+1 :green_heart: | mvninstall | 31m 12s | trunk passed | |
+1 :green_heart: | compile | 4m 18s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | compile | 3m 32s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | checkstyle | 1m 30s | trunk passed | |
+1 :green_heart: | mvnsite | 2m 4s | trunk passed | |
+1 :green_heart: | javadoc | 1m 53s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | javadoc | 1m 37s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | spotbugs | 3m 52s | trunk passed | |
+1 :green_heart: | shadedclient | 24m 58s | branch has no errors when building and testing our client artifacts. | |
_ Patch Compile Tests _ | ||||
+0 :ok: | mvndep | 0m 28s | Maven dependency ordering for patch | |
+1 :green_heart: | mvninstall | 1m 34s | the patch passed | |
+1 :green_heart: | compile | 4m 0s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | cc | 4m 0s | the patch passed | |
-1 :x: | javac | 4m 0s | /results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 3 new + 447 unchanged - 0 fixed = 450 total (was 447) |
+1 :green_heart: | compile | 3m 18s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | cc | 3m 18s | the patch passed | |
-1 :x: | javac | 3m 18s | /results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new + 371 unchanged - 0 fixed = 374 total (was 371) |
+1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
-0 :warning: | checkstyle | 1m 15s | /results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 18 new + 12 unchanged - 1 fixed = 30 total (was 13) |
+1 :green_heart: | mvnsite | 1m 42s | the patch passed | |
+1 :green_heart: | javadoc | 1m 24s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
+1 :green_heart: | javadoc | 1m 19s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
+1 :green_heart: | spotbugs | 3m 43s | the patch passed | |
+1 :green_heart: | shadedclient | 24m 35s | patch has no errors when building and testing our client artifacts. | |
_ Other Tests _ | ||||
-1 :x: | unit | 2m 52s | /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt | hadoop-yarn-server-common in the patch passed. |
+1 :green_heart: | unit | 123m 55s | hadoop-yarn-server-resourcemanager in the patch passed. | |
+1 :green_heart: | asflicense | 0m 49s | The patch does not generate ASF License warnings. | |
262m 21s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.yarn.server.federation.policies.router.TestLoadBasedRouterPolicy |
Subsystem | Report/Notes |
---|---|
Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/6/artifact/out/Dockerfile |
GITHUB PR | https://github.com/apache/hadoop/pull/4632 |
Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat |
uname | Linux 9e8e819153cd 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | dev-support/bin/hadoop.sh |
git revision | trunk / 6161eff62d11a4c49acafdaff3002453e22ad227 |
Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/6/testReport/ |
Max. process+thread count | 939 (vs. ulimit of 5500) |
modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server |
Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4632/6/console |
versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
@slfan1989 this is a lot to review. Can we split it? Probably cleanup of the existing code and then one or two for the federation part.
@slfan1989 this is a lot to review. Can we split it? Probably cleanup of the existing code and then one or two for the federation part.
@goiri Thank you very much for your help reviewing the code, I will split it.
The overall implementation idea is as follows 1.Refactor Policy Code and Define getReservationHomeSubcluster. - YARN-11235 2.Maintains the state of all Reservations that have been submitted to the Federated Cluster. MemoryStore - YARN-11236 ZKStore/SQLStore - YARN-7615 3.Support getNewReservation, submitReservation, updateReservation, deleteReservation API's for Federation. - YARN-11177 4.[Federation] Add createNewReservation, submitReservation, updateReservation, deleteReservation, listReservation REST APIs for Router. - YARN-11226
I uploaded a design document in YARN-5871, this is the V1 version (very simple), I will continue to add the content of the document, but it basically describes the work to be done by YARN-11235(https://github.com/apache/hadoop/pull/4656), I will upload the V2 version of the document as soon as possible.