hbase
hbase copied to clipboard
apache
HBASE-26055 Master local region "table" should be considered a system…
… table
The master local region is not a typical Table/Region, however, we should consider it a system table when creating the TableName so that if we call isSystemTable() it returns true.
One open question with this one: Should we prevent a user from creating a "master" namespace? Currently createNamespace doesn't fail for system namespaces (because it's assumed that the namespace already exists). In this case, the namespace doesn't exist, but the master namespace prefix is being used for a system table.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 1m 7s | Docker mode activated. |
| _ Prechecks _ | |||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 51s | master passed |
| +1 :green_heart: | compile | 4m 4s | master passed |
| +1 :green_heart: | checkstyle | 1m 28s | master passed |
| +1 :green_heart: | spotbugs | 2m 47s | master passed |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 37s | the patch passed |
| +1 :green_heart: | compile | 4m 0s | the patch passed |
| +1 :green_heart: | javac | 4m 0s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 27s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 18m 24s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 3m 9s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. |
| 53m 6s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3447 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ea775843c42e 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 147b030c1f |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Max. process+thread count | 96 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
The master namespace logic looks good to me, but I have a minor question, do you see anywhere we have any procedure or cleanup logic delete the master namespace ?
Should we prevent a user from creating a "master" namespace? Currently createNamespace doesn't fail for system namespaces (because it's assumed that the namespace already exists)
I found below that the prepareCreate as part fo the CreateNamespaceProcedure should fail if the namespace already exists, and you're right that it didn't protect the system namespace(s).
so, can you explain in what case the system namespaces in the meta table before it comes online would have the pre-existed namespaces? is it from a previous version ?
https://github.com/apache/hbase/blob/202b17f4fc3229e91d584837776d53ed3e2e8adb/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateNamespaceProcedure.java#L180-L193
Do you guys face any problems?
The master local region is not a typical table, so in general it should not be conflict with any real tables or regions. You could have a user level table called master:local, no problem.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 26s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 15s | master passed |
| +1 :green_heart: | compile | 1m 24s | master passed |
| +1 :green_heart: | shadedjars | 9m 6s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 58s | master passed |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 57s | the patch passed |
| +1 :green_heart: | compile | 1m 25s | the patch passed |
| +1 :green_heart: | javac | 1m 25s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 0s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 56s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 2m 2s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 184m 37s | hbase-server in the patch passed. |
| 220m 43s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3447 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 384ee11222a9 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 147b030c1f |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/testReport/ |
| Max. process+thread count | 2223 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 1m 5s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 5m 0s | master passed |
| +1 :green_heart: | compile | 1m 45s | master passed |
| +1 :green_heart: | shadedjars | 9m 5s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 6s | master passed |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 56s | the patch passed |
| +1 :green_heart: | compile | 1m 43s | the patch passed |
| +1 :green_heart: | javac | 1m 43s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 10s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 4s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 2m 11s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 183m 30s | hbase-server in the patch passed. |
| 223m 44s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3447 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 7830fd98ca57 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 147b030c1f |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/testReport/ |
| Max. process+thread count | 2429 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
Do you guys face any problems?
The master local region is not a typical table, so in general it should not be conflict with any real tables or regions. You could have a user level table called master:local, no problem.
I agree with @Apache9. I am kind of bit sceptical about making master local region as system table region. I might be missing something but does this improve any particular usecase? On the other hand, if this has some significant advantage, perhaps we can make it optional feature?
@Apache9 @virajjasani Sorry, I was out for the holiday. I understand that this is a special region and it does not have a corresponding table/namespace created for it. We do have an internal patch/feature that makes use of the system table designation (see HBASE-18477). We block writes/updates for user tables, but not system tables (HBASE-18775). Without this flag being set, writes to the master local region will be blocked with our feature. Originally the system table status was stored in the HRegionInfo, but now only the Table knows if it is a system table or not.
I can add a hack/hardcode a check for the master local region. However, it still seems like this is technically a system region? In that case, why would we not denote it as a system region? It certainly is not in the user's domain, it is an internal HBase region.
I'm fine with not adding this to the reserved NS list/restricting creating the NS, but to me it still seems like it is a bug that this "table" (really region) isn't being considered a system table.
@Apache9 @virajjasani Sorry, I was out for the holiday. I understand that this is a special region and it does not have a corresponding table/namespace created for it. We do have an internal patch/feature that makes use of the system table designation (see HBASE-18477). We block writes/updates for user tables, but not system tables (HBASE-18775). Without this flag being set, writes to the master local region will be blocked with our feature. Originally the system table status was stored in the HRegionInfo, but now only the Table knows if it is a system table or not.
I can add a hack/hardcode a check for the master local region. However, it still seems like this is technically a system region? In that case, why would we not denote it as a system region? It certainly is not in the user's domain, it is an internal HBase region.
I'm fine with not adding this to the reserved NS list/restricting creating the NS, but to me it still seems like it is a bug that this "table" (really region) isn't being considered a system table.
It is not a table actually... Neither a user table nor a system table...
So I suggest we should have a way to bypass the newly added logic, and we set this flag or config when constructing the master local region.
Thanks.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 25s | Docker mode activated. |
| _ Prechecks _ | |||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 7s | master passed |
| +1 :green_heart: | compile | 4m 12s | master passed |
| +1 :green_heart: | checkstyle | 1m 29s | master passed |
| +1 :green_heart: | spotbugs | 2m 50s | master passed |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 52s | the patch passed |
| +1 :green_heart: | compile | 4m 7s | the patch passed |
| +1 :green_heart: | javac | 4m 7s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 26s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 19m 39s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotbugs | 3m 19s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | asflicense | 0m 28s | The patch does not generate ASF License warnings. |
| 55m 4s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3447 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux d82a8b7b2c88 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f6348d4100 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Max. process+thread count | 96 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 26s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 45s | master passed |
| +1 :green_heart: | compile | 1m 42s | master passed |
| +1 :green_heart: | shadedjars | 8m 29s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 9s | master passed |
| _ Patch Compile Tests _ | |||
| +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 33s | the patch passed |
| +1 :green_heart: | compile | 1m 42s | the patch passed |
| +1 :green_heart: | javac | 1m 42s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 25s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 8s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 2m 5s | hbase-common in the patch passed. |
| -1 :x: | unit | 140m 6s | hbase-server in the patch failed. |
| 177m 27s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3447 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux e546837b8e7f 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / f6348d4100 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/testReport/ |
| Max. process+thread count | 3715 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.