hbase
hbase copied to clipboard
apache
HBASE-29715: AssignmentManager is trying to pick up the active cluster's tables before refreshing meta and hfiles
https://issues.apache.org/jira/browse/HBASE-29715
In a read-replica setup, two or more clusters share the same filesystem. This means the replica cluster can pick up tables created by the active cluster during metrics collection, even when the replica cluster has not had its metadata and HFiles refreshed yet. PR https://github.com/apache/hbase/pull/7304 introduced some logic to the TABLE_TO_REGIONS_COUNT case in HMaster.getClusterMetricsWithoutCoprocessor() that skips metrics collection for foreign meta tables.
This pull request broadens this logic to skip all foreign tables. This change will prevent the replica cluster from logging something like the following:
2025-11-14T03:22:08,098 ERROR [master/hbase-docker-2:16000.Chore.1] master.TableStateManager: Unable to get table testTable1 state
org.apache.hadoop.hbase.TableNotFoundException: No state found for testTable1
at org.apache.hadoop.hbase.master.TableStateManager.getTableState(TableStateManager.java:141) ~[hbase-server-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.master.TableStateManager.isTableState(TableStateManager.java:79) ~[hbase-server-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.master.assignment.AssignmentManager.isTableDisabled(AssignmentManager.java:549) ~[hbase-server-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.master.assignment.AssignmentManager.getRegionStatesCount(AssignmentManager.java:2637) ~[hbase-server-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.master.HMaster.getClusterMetricsWithoutCoprocessor(HMaster.java:3130) ~[hbase-server-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.master.HMaster.getClusterMetricsWithoutCoprocessor(HMaster.java:3017) ~[hbase-server-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.master.balancer.ClusterStatusChore.chore(ClusterStatusChore.java:47) ~[hbase-server-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:161) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?]
at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
at java.lang.Thread.run(Thread.java:840) ~[?:?]
With this PR's changes, the following will be logged instead:
2025-11-20T04:17:29,535 INFO [master/hbase-docker-2:16000.Chore.1] master.HMaster: Skipping foreign table testTable1 in cluster metrics
I noticed PR #7304 also made changes to FSTableDescriptors.java and HbckChore.java via FSUtils.isLocalMetaTable(). Currently, this PR does not make any changes to those files.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 35s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | |
| _ HBASE-29081 Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 3m 58s | HBASE-29081 passed | |
| +1 :green_heart: | compile | 3m 41s | HBASE-29081 passed | |
| -0 :warning: | checkstyle | 0m 18s | /buildtool-branch-checkstyle-hbase-server.txt | The patch fails to run checkstyle in hbase-server |
| +1 :green_heart: | spotbugs | 1m 46s | HBASE-29081 passed | |
| +1 :green_heart: | spotless | 0m 55s | branch has no errors when running spotless:check. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 3m 27s | the patch passed | |
| +1 :green_heart: | compile | 3m 37s | the patch passed | |
| +1 :green_heart: | javac | 3m 37s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 0m 15s | /buildtool-patch-checkstyle-hbase-server.txt | The patch fails to run checkstyle in hbase-server |
| +1 :green_heart: | spotbugs | 1m 51s | the patch passed | |
| +1 :green_heart: | hadoopcheck | 12m 43s | Patch does not cause any errors with Hadoop 3.3.6 3.4.0. | |
| +1 :green_heart: | spotless | 0m 50s | patch has no errors when running spotless:check. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | |
| 42m 1s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7474/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7474 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 40eab400b95b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-29081 / 7c9c85a93cb07b487050ce0bca2eba67a58d6bd9 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 83 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7474/1/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 2m 18s | Docker mode activated. | |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck | |
| _ Prechecks _ | ||||
| _ HBASE-29081 Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 4m 55s | HBASE-29081 passed | |
| +1 :green_heart: | compile | 1m 14s | HBASE-29081 passed | |
| +1 :green_heart: | javadoc | 0m 36s | HBASE-29081 passed | |
| +1 :green_heart: | shadedjars | 6m 58s | branch has no errors when building our shaded downstream artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 4m 23s | the patch passed | |
| +1 :green_heart: | compile | 1m 15s | the patch passed | |
| +1 :green_heart: | javac | 1m 15s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 33s | the patch passed | |
| +1 :green_heart: | shadedjars | 6m 55s | patch has no errors when building our shaded downstream artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 538m 49s | /patch-unit-hbase-server.txt | hbase-server in the patch failed. |
| 572m 25s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7474/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7474 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 8ced6ccd755a 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-29081 / 7c9c85a93cb07b487050ce0bca2eba67a58d6bd9 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7474/1/testReport/ |
| Max. process+thread count | 2810 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7474/1/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
@Kota-SH @kgeisz Are the unit test failures related? Can we move on with merging this PR?
@anmolnar The failed unit tests are passing for me as well when I run them locally.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 34s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | |
| _ HBASE-29081 Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 3m 39s | HBASE-29081 passed | |
| +1 :green_heart: | compile | 3m 38s | HBASE-29081 passed | |
| -0 :warning: | checkstyle | 0m 15s | /buildtool-branch-checkstyle-hbase-server.txt | The patch fails to run checkstyle in hbase-server |
| +1 :green_heart: | spotbugs | 1m 40s | HBASE-29081 passed | |
| +1 :green_heart: | spotless | 0m 52s | branch has no errors when running spotless:check. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 3m 16s | the patch passed | |
| +1 :green_heart: | compile | 3m 31s | the patch passed | |
| +1 :green_heart: | javac | 3m 31s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 0m 13s | /buildtool-patch-checkstyle-hbase-server.txt | The patch fails to run checkstyle in hbase-server |
| +1 :green_heart: | spotbugs | 1m 42s | the patch passed | |
| +1 :green_heart: | hadoopcheck | 12m 22s | Patch does not cause any errors with Hadoop 3.3.6 3.4.0. | |
| -1 :x: | spotless | 0m 39s | patch has 26 errors when running spotless:check, run spotless:apply to fix. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | |
| 40m 17s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7474/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7474 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 8e872dc59110 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-29081 / 35e5cbac228b0842d46ec8436bee41d7f100f34e |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7474/2/artifact/yetus-general-check/output/patch-spotless.txt |
| Max. process+thread count | 85 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7474/2/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 1m 22s | Docker mode activated. | |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck | |
| _ Prechecks _ | ||||
| _ HBASE-29081 Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 2m 47s | HBASE-29081 passed | |
| +1 :green_heart: | compile | 0m 43s | HBASE-29081 passed | |
| +1 :green_heart: | javadoc | 0m 23s | HBASE-29081 passed | |
| +1 :green_heart: | shadedjars | 4m 29s | branch has no errors when building our shaded downstream artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 2m 15s | the patch passed | |
| +1 :green_heart: | compile | 0m 44s | the patch passed | |
| +1 :green_heart: | javac | 0m 44s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 20s | the patch passed | |
| +1 :green_heart: | shadedjars | 4m 26s | patch has no errors when building our shaded downstream artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 229m 47s | /patch-unit-hbase-server.txt | hbase-server in the patch failed. |
| 250m 58s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7474/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7474 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux fee7bb96f70d 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-29081 / 35e5cbac228b0842d46ec8436bee41d7f100f34e |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7474/2/testReport/ |
| Max. process+thread count | 6150 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7474/2/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
@kgeisz please create a follow up to add a test for fresh replica setup that loads from filesystem (instead of the currently one that is loaded from cache.)
