hbase
hbase copied to clipboard
apache
HBASE-27389 Add cost function in balancer to consider the cost of bui…
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 39s | Docker mode activated. |
| _ Prechecks _ | |||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | prototool | 0m 0s | prototool was not available. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 26s | master passed |
| +1 :green_heart: | compile | 3m 57s | master passed |
| +1 :green_heart: | checkstyle | 0m 53s | master passed |
| +1 :green_heart: | spotless | 0m 38s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 5m 4s | master passed |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 25s | the patch passed |
| +1 :green_heart: | compile | 5m 7s | the patch passed |
| +1 :green_heart: | cc | 5m 7s | the patch passed |
| +1 :green_heart: | javac | 5m 7s | the patch passed |
| -0 :warning: | checkstyle | 0m 9s | hbase-balancer: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) |
| -0 :warning: | checkstyle | 0m 31s | hbase-server: The patch generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| -1 :x: | hadoopcheck | 5m 44s | The patch causes 10 errors with Hadoop v3.3.4. |
| +1 :green_heart: | hbaseprotoc | 2m 1s | the patch passed |
| -1 :x: | spotless | 0m 21s | patch has 33 errors when running spotless:check, run spotless:apply to fix. |
| -1 :x: | spotbugs | 2m 7s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| _ Other Tests _ | |||
| +1 :green_heart: | asflicense | 1m 2s | The patch does not generate ASF License warnings. |
| 41m 28s |
| Reason | Tests |
|---|---|
| FindBugs | module:hbase-server |
| Integral division result cast to double or float in org.apache.hadoop.hbase.regionserver.HRegionServer.createRegionLoad(HRegion, ClusterStatusProtos$RegionLoad$Builder, HBaseProtos$RegionSpecifier$Builder) At HRegionServer.java:double or float in org.apache.hadoop.hbase.regionserver.HRegionServer.createRegionLoad(HRegion, ClusterStatusProtos$RegionLoad$Builder, HBaseProtos$RegionSpecifier$Builder) At HRegionServer.java:[line 1546] |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile cc hbaseprotoc prototool |
| uname | Linux f4625729b680 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 748cad655c |
| Default Java | Temurin-1.8.0_345-b01 |
| checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-balancer.txt |
| checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| hadoopcheck | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/artifact/yetus-general-check/output/patch-javac-3.3.4.txt |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/artifact/yetus-general-check/output/patch-spotless.txt |
| spotbugs | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html |
| Max. process+thread count | 64 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.7.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 38s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 24s | master passed |
| +1 :green_heart: | compile | 1m 34s | master passed |
| +1 :green_heart: | shadedjars | 4m 8s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 54s | master passed |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 9s | the patch passed |
| +1 :green_heart: | compile | 1m 29s | the patch passed |
| +1 :green_heart: | javac | 1m 29s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 5s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 54s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 0m 25s | hbase-protocol-shaded in the patch passed. |
| +1 :green_heart: | unit | 1m 9s | hbase-client in the patch passed. |
| -1 :x: | unit | 10m 49s | hbase-balancer in the patch failed. |
| -1 :x: | unit | 13m 33s | hbase-server in the patch failed. |
| 46m 33s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 9bb1c29a6ec3 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 748cad655c |
| Default Java | Temurin-1.8.0_345-b01 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-balancer.txt |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/testReport/ |
| Max. process+thread count | 899 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 51s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 38s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 37s | master passed |
| +1 :green_heart: | compile | 1m 40s | master passed |
| +1 :green_heart: | shadedjars | 3m 56s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 58s | master passed |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 22s | the patch passed |
| +1 :green_heart: | compile | 1m 41s | the patch passed |
| +1 :green_heart: | javac | 1m 41s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 53s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 56s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 0m 31s | hbase-protocol-shaded in the patch passed. |
| +1 :green_heart: | unit | 1m 19s | hbase-client in the patch passed. |
| -1 :x: | unit | 12m 1s | hbase-balancer in the patch failed. |
| -1 :x: | unit | 14m 53s | hbase-server in the patch failed. |
| 49m 55s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 9329e847d468 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 748cad655c |
| Default Java | Eclipse Adoptium-11.0.16.1+1 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-balancer.txt |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/testReport/ |
| Max. process+thread count | 961 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 1m 41s | Docker mode activated. |
| _ Prechecks _ | |||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | prototool | 0m 1s | prototool was not available. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 21s | master passed |
| +1 :green_heart: | compile | 6m 6s | master passed |
| +1 :green_heart: | checkstyle | 1m 21s | master passed |
| +1 :green_heart: | spotless | 0m 52s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 7m 36s | master passed |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 13s | the patch passed |
| +1 :green_heart: | compile | 3m 54s | the patch passed |
| +1 :green_heart: | cc | 3m 54s | the patch passed |
| -0 :warning: | javac | 2m 12s | hbase-server generated 1 new + 192 unchanged - 1 fixed = 193 total (was 193) |
| +1 :green_heart: | checkstyle | 0m 51s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 8m 3s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. |
| +1 :green_heart: | hbaseprotoc | 1m 25s | the patch passed |
| +1 :green_heart: | spotless | 0m 36s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 5m 27s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. |
| 50m 4s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile cc hbaseprotoc prototool |
| uname | Linux 41a300156642 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 748cad655c |
| Default Java | Temurin-1.8.0_345-b01 |
| javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/2/artifact/yetus-general-check/output/diff-compile-javac-hbase-server.txt |
| Max. process+thread count | 64 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/2/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.7.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 36s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 4s | master passed |
| +1 :green_heart: | compile | 1m 26s | master passed |
| +1 :green_heart: | shadedjars | 4m 7s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 52s | master passed |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 5s | the patch passed |
| +1 :green_heart: | compile | 1m 27s | the patch passed |
| +1 :green_heart: | javac | 1m 27s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 5s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 52s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 0m 25s | hbase-protocol-shaded in the patch passed. |
| +1 :green_heart: | unit | 1m 9s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 8m 48s | hbase-balancer in the patch passed. |
| +1 :green_heart: | unit | 202m 22s | hbase-server in the patch passed. |
| 232m 23s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux e7ca5ee4f10e 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 748cad655c |
| Default Java | Temurin-1.8.0_345-b01 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/2/testReport/ |
| Max. process+thread count | 2579 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 1m 15s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 48s | master passed |
| +1 :green_heart: | compile | 2m 0s | master passed |
| +1 :green_heart: | shadedjars | 3m 55s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 8s | master passed |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 35s | the patch passed |
| +1 :green_heart: | compile | 2m 0s | the patch passed |
| +1 :green_heart: | javac | 2m 0s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 45s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 7s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 0m 38s | hbase-protocol-shaded in the patch passed. |
| +1 :green_heart: | unit | 1m 27s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 11m 46s | hbase-balancer in the patch passed. |
| +1 :green_heart: | unit | 204m 10s | hbase-server in the patch passed. |
| 241m 4s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 650e5b89e9e0 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 748cad655c |
| Default Java | Eclipse Adoptium-11.0.16.1+1 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/2/testReport/ |
| Max. process+thread count | 2708 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
Mind explaining more about the algorithm here?
I guess the problem here is that, when moving a region from rs A to rs B, the block cache on A is useless now and then on rs B, we need to reload the block cache of this region, and it will evict other regions data?
What is the algo here to measure this cost?
Thanks.
Mind explaining more about the algorithm here?
I guess the problem here is that, when moving a region from rs A to rs B, the block cache on A is useless now and then on rs B, we need to reload the block cache of this region, and it will evict other regions data?
What is the algo here to measure this cost?
Thanks.
Hello,
All the region servers maintain a list of all the HFiles which are already cached. This change was done as part of HBASE-27313. The current JIRA uses this information to find out the ratio of files already cached to the total number of files in the region server and puts this information in the region specific metric.
The stochastic load balancer uses this information to find out the cost of moving a region from one region server to the other by comparing the ratio of files already prefetched. Higher the ratio of files prefetched on a region server, lower the cost of moving the region. In addition to this information, the patch also considers the region size to measure the weighted cost moving the region.
I do not fully understand why higher prefetched ratio leads to lower moving cost, if a region's hfiles have all been cached, after moving we need to fetched all the hfiles again, which is costly?
I do not fully understand why higher prefetched ratio leads to lower moving cost, if a region's hfiles have all been cached, after moving we need to fetched all the hfiles again, which is costly?
If we are moving a region from server A to server B, if the prefetch ratio on B is higher than that on A, then the cost of moving the region is low. During region movement, there is a likelihood that the file is already prefetched on server B and hence there will be no need to prefetch it again. The changes done in HBASE-27313 already maintain a list of files which are already prefetched on the server.
Also, does this new cost function always be enabled by default on the StochasticLoadBalancer? The PrefetchCacheCostFunction will be enabled if the configuration "hbase.prefetch.file.list.path" is set. In addition to that, it will have it's own configuration "hbase.master.balancer.stochastic.prefetchCacheCost" that will define the multiplier value for this cost function. This is covered in the new patch that I will upload soon. If either "hbase.prefetch.file.list.path" is not set or "hbase.master.balancer.stochastic.prefetchCacheCost" is set to 0, the cost function will not be enabled. The default value of "hbase.master.balancer.stochastic.prefetchCacheCost" will be > 0 which will mean that the cost function will be enabled by default given that the configuration "hbase.prefetch.file.list.path" is set.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 38s | Docker mode activated. |
| _ Prechecks _ | |||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | prototool | 0m 0s | prototool was not available. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 49s | master passed |
| +1 :green_heart: | compile | 4m 28s | master passed |
| +1 :green_heart: | checkstyle | 1m 12s | master passed |
| +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 5m 17s | master passed |
| -0 :warning: | patch | 1m 51s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 31s | the patch passed |
| +1 :green_heart: | compile | 4m 26s | the patch passed |
| +1 :green_heart: | cc | 4m 26s | the patch passed |
| -0 :warning: | javac | 0m 17s | hbase-balancer generated 2 new + 14 unchanged - 0 fixed = 16 total (was 14) |
| -0 :warning: | checkstyle | 0m 8s | hbase-balancer: The patch generated 6 new + 1 unchanged - 0 fixed = 7 total (was 1) |
| -0 :warning: | checkstyle | 0m 31s | hbase-server: The patch generated 1 new + 9 unchanged - 0 fixed = 10 total (was 9) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 9m 53s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. |
| +1 :green_heart: | hbaseprotoc | 1m 41s | the patch passed |
| -1 :x: | spotless | 0m 18s | patch has 19 errors when running spotless:check, run spotless:apply to fix. |
| +1 :green_heart: | spotbugs | 5m 56s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | asflicense | 0m 37s | The patch does not generate ASF License warnings. |
| 49m 15s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile cc hbaseprotoc prototool |
| uname | Linux 7eca0ea16fac 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ad8f28e297 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/3/artifact/yetus-general-check/output/diff-compile-javac-hbase-balancer.txt |
| checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-balancer.txt |
| checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/3/artifact/yetus-general-check/output/patch-spotless.txt |
| Max. process+thread count | 80 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/3/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 1m 18s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 20s | master passed |
| +1 :green_heart: | compile | 1m 39s | master passed |
| +1 :green_heart: | shadedjars | 4m 0s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 4s | master passed |
| -0 :warning: | patch | 5m 35s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 6s | the patch passed |
| +1 :green_heart: | compile | 1m 39s | the patch passed |
| +1 :green_heart: | javac | 1m 39s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 59s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 7s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 0m 25s | hbase-protocol-shaded in the patch passed. |
| +1 :green_heart: | unit | 1m 41s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 9s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 9m 54s | hbase-balancer in the patch passed. |
| +1 :green_heart: | unit | 206m 55s | hbase-server in the patch passed. |
| 244m 57s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux d8de9ea15f48 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ad8f28e297 |
| Default Java | Temurin-1.8.0_352-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/3/testReport/ |
| Max. process+thread count | 2671 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/3/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 24s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 57s | master passed |
| +1 :green_heart: | compile | 3m 28s | master passed |
| +1 :green_heart: | shadedjars | 5m 44s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 36s | master passed |
| -0 :warning: | patch | 7m 55s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 19s | the patch passed |
| +1 :green_heart: | compile | 2m 14s | the patch passed |
| +1 :green_heart: | javac | 2m 14s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 45s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 20s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 0m 39s | hbase-protocol-shaded in the patch passed. |
| +1 :green_heart: | unit | 2m 13s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 22s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 9m 34s | hbase-balancer in the patch passed. |
| +1 :green_heart: | unit | 207m 47s | hbase-server in the patch passed. |
| 253m 17s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 791f06bbc546 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ad8f28e297 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/3/testReport/ |
| Max. process+thread count | 2733 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/3/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 39s | Docker mode activated. |
| _ Prechecks _ | |||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | prototool | 0m 0s | prototool was not available. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 33s | master passed |
| +1 :green_heart: | compile | 4m 29s | master passed |
| +1 :green_heart: | checkstyle | 1m 11s | master passed |
| +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 5m 10s | master passed |
| -0 :warning: | patch | 1m 52s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 31s | the patch passed |
| +1 :green_heart: | compile | 4m 26s | the patch passed |
| +1 :green_heart: | cc | 4m 26s | the patch passed |
| -0 :warning: | javac | 0m 18s | hbase-balancer generated 2 new + 14 unchanged - 0 fixed = 16 total (was 14) |
| +1 :green_heart: | checkstyle | 1m 12s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 9m 44s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. |
| +1 :green_heart: | hbaseprotoc | 1m 40s | the patch passed |
| +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 5m 52s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. |
| 49m 2s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile cc hbaseprotoc prototool |
| uname | Linux f6757abb2e69 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ad8f28e297 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/4/artifact/yetus-general-check/output/diff-compile-javac-hbase-balancer.txt |
| Max. process+thread count | 81 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/4/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 39s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 40s | master passed |
| +1 :green_heart: | compile | 2m 4s | master passed |
| +1 :green_heart: | shadedjars | 3m 58s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 7s | master passed |
| -0 :warning: | patch | 5m 37s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 24s | the patch passed |
| +1 :green_heart: | compile | 1m 57s | the patch passed |
| +1 :green_heart: | javac | 1m 57s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 57s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 6s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 0m 33s | hbase-protocol-shaded in the patch passed. |
| +1 :green_heart: | unit | 2m 0s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 20s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 9m 49s | hbase-balancer in the patch passed. |
| +1 :green_heart: | unit | 202m 59s | hbase-server in the patch passed. |
| 242m 7s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux b09d9fa6c0df 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ad8f28e297 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/4/testReport/ |
| Max. process+thread count | 2632 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/4/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 39s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 21s | master passed |
| +1 :green_heart: | compile | 1m 41s | master passed |
| +1 :green_heart: | shadedjars | 4m 7s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 4s | master passed |
| -0 :warning: | patch | 5m 43s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 9s | the patch passed |
| +1 :green_heart: | compile | 1m 40s | the patch passed |
| +1 :green_heart: | javac | 1m 40s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 5s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 3s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 0m 25s | hbase-protocol-shaded in the patch passed. |
| +1 :green_heart: | unit | 1m 41s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 9s | hbase-client in the patch passed. |
| -1 :x: | unit | 20m 1s | hbase-balancer in the patch failed. |
| -1 :x: | unit | 207m 28s | hbase-server in the patch failed. |
| 254m 53s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 74c6d5416c80 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ad8f28e297 |
| Default Java | Temurin-1.8.0_352-b08 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-balancer.txt |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/4/testReport/ |
| Max. process+thread count | 2647 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/4/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 1m 1s | Docker mode activated. |
| _ Prechecks _ | |||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | prototool | 0m 0s | prototool was not available. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 17s | master passed |
| +1 :green_heart: | compile | 5m 17s | master passed |
| +1 :green_heart: | checkstyle | 1m 26s | master passed |
| +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 6m 11s | master passed |
| -0 :warning: | patch | 2m 19s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 8s | the patch passed |
| +1 :green_heart: | compile | 5m 13s | the patch passed |
| +1 :green_heart: | cc | 5m 13s | the patch passed |
| -0 :warning: | javac | 0m 22s | hbase-balancer generated 2 new + 14 unchanged - 0 fixed = 16 total (was 14) |
| +1 :green_heart: | checkstyle | 1m 31s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 10m 49s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. |
| +1 :green_heart: | hbaseprotoc | 2m 8s | the patch passed |
| +1 :green_heart: | spotless | 0m 48s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 7m 6s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | asflicense | 0m 42s | The patch does not generate ASF License warnings. |
| 57m 52s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/5/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile cc hbaseprotoc prototool |
| uname | Linux 577325c8d88b 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ad8f28e297 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/5/artifact/yetus-general-check/output/diff-compile-javac-hbase-balancer.txt |
| Max. process+thread count | 81 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/5/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 1m 7s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 37s | master passed |
| +1 :green_heart: | compile | 2m 18s | master passed |
| +1 :green_heart: | shadedjars | 3m 51s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 21s | master passed |
| -0 :warning: | patch | 5m 49s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 30s | the patch passed |
| +1 :green_heart: | compile | 2m 16s | the patch passed |
| +1 :green_heart: | javac | 2m 16s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 46s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 18s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 0m 38s | hbase-protocol-shaded in the patch passed. |
| +1 :green_heart: | unit | 2m 5s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 27s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 9m 42s | hbase-balancer in the patch passed. |
| +1 :green_heart: | unit | 210m 32s | hbase-server in the patch passed. |
| 251m 13s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux d31127b6e883 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ad8f28e297 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/5/testReport/ |
| Max. process+thread count | 2721 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/5/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 1m 12s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 2m 14s | master passed |
| +1 :green_heart: | compile | 1m 42s | master passed |
| +1 :green_heart: | shadedjars | 4m 31s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 15s | master passed |
| -0 :warning: | patch | 6m 21s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 18s | the patch passed |
| +1 :green_heart: | compile | 1m 45s | the patch passed |
| +1 :green_heart: | javac | 1m 45s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 16s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 2s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 0m 24s | hbase-protocol-shaded in the patch passed. |
| +1 :green_heart: | unit | 1m 47s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 36s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 10m 12s | hbase-balancer in the patch passed. |
| -1 :x: | unit | 212m 48s | hbase-server in the patch failed. |
| 252m 47s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 42d364c15fc4 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ad8f28e297 |
| Default Java | Temurin-1.8.0_352-b08 |
| unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/5/testReport/ |
| Max. process+thread count | 2883 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/5/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
The test reported failing in the PR seems to be failing inconsistently as it passed when tested on a local build. Also, the failure do not seem to be related to the code changes in this PR.
Attempting to rebuild this PR once again.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 41s | Docker mode activated. |
| _ Prechecks _ | |||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | prototool | 0m 0s | prototool was not available. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 44s | master passed |
| +1 :green_heart: | compile | 4m 24s | master passed |
| +1 :green_heart: | checkstyle | 1m 9s | master passed |
| +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 5m 16s | master passed |
| -0 :warning: | patch | 1m 52s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 20s | the patch passed |
| +1 :green_heart: | compile | 4m 25s | the patch passed |
| +1 :green_heart: | cc | 4m 25s | the patch passed |
| -0 :warning: | javac | 0m 18s | hbase-balancer generated 2 new + 14 unchanged - 0 fixed = 16 total (was 14) |
| +1 :green_heart: | checkstyle | 1m 12s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 13m 33s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. |
| +1 :green_heart: | hbaseprotoc | 5m 40s | the patch passed |
| +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 5m 49s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | asflicense | 0m 37s | The patch does not generate ASF License warnings. |
| 60m 30s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/6/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile cc hbaseprotoc prototool |
| uname | Linux 1dd8a772fa6d 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 83d450d5b5 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/6/artifact/yetus-general-check/output/diff-compile-javac-hbase-balancer.txt |
| Max. process+thread count | 86 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/6/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 44s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 37s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 3s | master passed |
| +1 :green_heart: | compile | 1m 42s | master passed |
| +1 :green_heart: | shadedjars | 4m 39s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 5s | master passed |
| -0 :warning: | patch | 6m 15s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 43s | the patch passed |
| +1 :green_heart: | compile | 1m 42s | the patch passed |
| +1 :green_heart: | javac | 1m 42s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 41s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 7s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 0m 28s | hbase-protocol-shaded in the patch passed. |
| +1 :green_heart: | unit | 1m 42s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 9s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 9m 53s | hbase-balancer in the patch passed. |
| +1 :green_heart: | unit | 210m 57s | hbase-server in the patch passed. |
| 250m 57s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 9690dcd94577 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 83d450d5b5 |
| Default Java | Temurin-1.8.0_352-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/6/testReport/ |
| Max. process+thread count | 2641 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/6/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 1m 5s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 25s | master passed |
| +1 :green_heart: | compile | 2m 15s | master passed |
| +1 :green_heart: | shadedjars | 4m 23s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 19s | master passed |
| -0 :warning: | patch | 6m 21s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 15s | the patch passed |
| +1 :green_heart: | compile | 2m 15s | the patch passed |
| +1 :green_heart: | javac | 2m 15s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 20s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 19s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 0m 37s | hbase-protocol-shaded in the patch passed. |
| +1 :green_heart: | unit | 2m 6s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 27s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 9m 30s | hbase-balancer in the patch passed. |
| +1 :green_heart: | unit | 232m 37s | hbase-server in the patch passed. |
| 275m 31s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4799 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 5db1539dcd1c 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 83d450d5b5 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/6/testReport/ |
| Max. process+thread count | 2415 (vs. ulimit of 30000) |
| modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4799/6/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
An update on testing. In addition to adding a unit test which considers different combinations of prefetch ratio on the currently hosting region server as well as historical prefetch ratio, these changes were tested on a real cluster. However, it was only tested for happy condition viz. if the prefetch ratio on a region server was good, then the region was not moved back to the old region server even though it was hosted and prefetched on the older region server in the part. This test was done by disabling all other cost functions by setting their multiplier values to 0. Every cost function provides a parameter which can be set to 0 to disable that particular cost function.
An update on testing. In addition to adding a unit test which considers different combinations of prefetch ratio on the currently hosting region server as well as historical prefetch ratio, these changes were tested on a real cluster. However, it was only tested for happy condition viz. if the prefetch ratio on a region server was good, then the region was not moved back to the old region server even though it was hosted and prefetched on the older region server in the part. This test was done by disabling all other cost functions by setting their multiplier values to 0. Every cost function provides a parameter which can be set to 0 to disable that particular cost function.
Thanks for the updates! I was looking at the existing functions and based on the respective multipliers default values, I guess region skewness and locality are currently considered the most relevant factors for StochasticLoadBalancer.
The prefetch ratio function would be most relevant for non-hdfs deployments, which then makes locality irrelevant. In those cases, prefetch and region skewness would be competing to prevail over the final balancer decision. We should also document this and mention that the behaviour is configurable via the function multiplier property.
