apache
HBASE-28962 Meta replication is inconsistent after startup when reusi…
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 5m 22s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | |
| _ branch-2.6 Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 3m 36s | branch-2.6 passed | |
| +1 :green_heart: | compile | 3m 3s | branch-2.6 passed | |
| +1 :green_heart: | checkstyle | 0m 38s | branch-2.6 passed | |
| +1 :green_heart: | spotbugs | 1m 36s | branch-2.6 passed | |
| +1 :green_heart: | spotless | 0m 47s | branch has no errors when running spotless:check. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 3m 10s | the patch passed | |
| +1 :green_heart: | compile | 2m 58s | the patch passed | |
| +1 :green_heart: | javac | 2m 58s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 0m 37s | the patch passed | |
| +1 :green_heart: | spotbugs | 1m 42s | the patch passed | |
| +1 :green_heart: | hadoopcheck | 17m 19s | Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.0. | |
| -1 :x: | spotless | 0m 37s | patch has 24 errors when running spotless:check, run spotless:apply to fix. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | |
| 46m 10s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7046/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7046 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux b894bdb24ec8 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2.6 / 74ee25f2eb97a1b0bef13e8d8a4269a9fbf575fd |
| Default Java | Eclipse Adoptium-11.0.23+9 |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7046/2/artifact/yetus-general-check/output/patch-spotless.txt |
| Max. process+thread count | 78 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7046/2/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 53s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | |
| _ branch-2.6 Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 3m 27s | branch-2.6 passed | |
| +1 :green_heart: | compile | 2m 58s | branch-2.6 passed | |
| +1 :green_heart: | checkstyle | 0m 38s | branch-2.6 passed | |
| +1 :green_heart: | spotbugs | 1m 38s | branch-2.6 passed | |
| +1 :green_heart: | spotless | 0m 49s | branch has no errors when running spotless:check. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 3m 6s | the patch passed | |
| +1 :green_heart: | compile | 2m 55s | the patch passed | |
| +1 :green_heart: | javac | 2m 55s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 0m 37s | the patch passed | |
| +1 :green_heart: | spotbugs | 1m 43s | the patch passed | |
| +1 :green_heart: | hadoopcheck | 17m 16s | Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.0. | |
| +1 :green_heart: | spotless | 0m 44s | patch has no errors when running spotless:check. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | |
| 38m 59s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7046/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7046 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux aef2cab960dd 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2.6 / 31d7b6bb18b30e3b286d8bbbf967dc8c8334e9cb |
| Default Java | Eclipse Adoptium-11.0.23+9 |
| Max. process+thread count | 78 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7046/3/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 46s | Docker mode activated. | |
| -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck | |
| _ Prechecks _ | ||||
| _ branch-2.6 Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 3m 20s | branch-2.6 passed | |
| +1 :green_heart: | compile | 0m 50s | branch-2.6 passed | |
| +1 :green_heart: | javadoc | 0m 26s | branch-2.6 passed | |
| +1 :green_heart: | shadedjars | 6m 36s | branch has no errors when building our shaded downstream artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 3m 11s | the patch passed | |
| +1 :green_heart: | compile | 0m 50s | the patch passed | |
| +1 :green_heart: | javac | 0m 50s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 26s | the patch passed | |
| +1 :green_heart: | shadedjars | 6m 36s | patch has no errors when building our shaded downstream artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 209m 39s | hbase-server in the patch passed. | |
| 236m 44s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7046/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7046 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux c4c6fcd6ce5b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2.6 / 31d7b6bb18b30e3b286d8bbbf967dc8c8334e9cb |
| Default Java | Eclipse Adoptium-11.0.23+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7046/3/testReport/ |
| Max. process+thread count | 4499 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7046/3/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 55s | Docker mode activated. | |
| -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck | |
| _ Prechecks _ | ||||
| _ branch-2.6 Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 4m 39s | branch-2.6 passed | |
| +1 :green_heart: | compile | 1m 9s | branch-2.6 passed | |
| +1 :green_heart: | javadoc | 0m 31s | branch-2.6 passed | |
| +1 :green_heart: | shadedjars | 6m 36s | branch has no errors when building our shaded downstream artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 3m 19s | the patch passed | |
| +1 :green_heart: | compile | 1m 0s | the patch passed | |
| +1 :green_heart: | javac | 1m 0s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 28s | the patch passed | |
| +1 :green_heart: | shadedjars | 6m 36s | patch has no errors when building our shaded downstream artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 215m 42s | hbase-server in the patch passed. | |
| 245m 24s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7046/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7046 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 5f6e7c505f0b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2.6 / 31d7b6bb18b30e3b286d8bbbf967dc8c8334e9cb |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7046/3/testReport/ |
| Max. process+thread count | 4419 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7046/3/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 1m 15s | Docker mode activated. | |
| -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck | |
| _ Prechecks _ | ||||
| _ branch-2.6 Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 3m 28s | branch-2.6 passed | |
| +1 :green_heart: | compile | 0m 42s | branch-2.6 passed | |
| +1 :green_heart: | javadoc | 0m 26s | branch-2.6 passed | |
| +1 :green_heart: | shadedjars | 5m 35s | branch has no errors when building our shaded downstream artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 2m 29s | the patch passed | |
| +1 :green_heart: | compile | 0m 41s | the patch passed | |
| +1 :green_heart: | javac | 0m 41s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 24s | the patch passed | |
| +1 :green_heart: | shadedjars | 5m 28s | patch has no errors when building our shaded downstream artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 225m 24s | hbase-server in the patch passed. | |
| 250m 14s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7046/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7046 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 67aebd5896f2 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2.6 / 31d7b6bb18b30e3b286d8bbbf967dc8c8334e9cb |
| Default Java | Temurin-1.8.0_412-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7046/3/testReport/ |
| Max. process+thread count | 4189 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7046/3/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
We run into this issue from time to time. When creating a cluster with storage location set to a storage location of an earlier cluster. Both clusters are set to use meta replication. Only the default replica gets assigned on the new cluster.
Do you think this change could cause any problems?
We run into this issue from time to time. When creating a cluster with storage location set to a storage location of an earlier cluster. Both clusters are set to use meta replication. Only the default replica gets assigned on the new cluster.
Do you think this change could cause any problems?
I agree that before identifying the root cause, you can apply some temporary approaches on your internal branches to 'fix' the problem.
But on the open source branch, I think we should find the root cause and then apply the 'correct' fix.
As I've said above, please check whether we have the master local region related changes in your internal branch.
And on the unit testing, please try delete everything on zookeeper to simulate your scenario. You can see HBASE-29292 and TestRecreateCluster on how to do this. If you can still produce the problem, we can check again whether we need the changes here.
Thanks.
Please do not merge this.
Checked the code on branch-2.6, we also load the meta replica locations from master local region, not zookeeper. So the comment here is incorrect
Please also revert the changes from existing branches, and let's review what is the root cause of this problem.
Thanks.
https://github.com/apache/hbase/blob/2f557ab02995fa54dba0acd85562eca18f3bc3de/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java#L316
@Apache9 is right, since HBASE-26193, present since 2.5.0, we don't rely on ZK for the region states anymore. @richardantal , have you still managed to reproduce this issue on a 2.5/2.6 based deployment?
Any updates here?
Thanks.
I'm going to revert this from the other branches. Here's the revert PR: https://github.com/apache/hbase/pull/7125