hbase icon indicating copy to clipboard operation
hbase copied to clipboard

HBASE-28837: Row Statistics Coprocessor

Open eab148 opened this issue 1 year ago • 27 comments

Design doc

HBase provides many configuration options to help administrators tune their tables and clusters for desired performance and reliability. However, it can be difficult for administrators to leverage these configuration options because they do not have a nuanced understanding of the shape of their data in HBase.

In this PR, we introduce a row statistics coprocessor to the hbase-examples module. This coprocessor allows administrators to collect statistics on the rows in their HBase tables as these rows compact. With more information about the shape of their data in HBase, administrators can leverage the available configuration options to unlock performance and reliability gains for their HBase tables/clusters.

(Thank you to Bryan Beaudreault for collaborating with me on this!)

Jira

cc @hgromer @krconv @rmdmattingly @ndimiduk @bbeaudreault

eab148 avatar Sep 30 '24 15:09 eab148

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 0s Docker mode activated.
-1 :x: patch 0m 6s https://github.com/apache/hbase/pull/6327 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.
Subsystem Report/Notes
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28842
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/1/console
versions git=2.17.1
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Sep 30 '24 17:09 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 0s Docker mode activated.
-1 :x: patch 0m 5s https://github.com/apache/hbase/pull/6327 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.
Subsystem Report/Notes
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28842
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/1/console
versions git=2.17.1
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Sep 30 '24 17:09 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 26s Docker mode activated.
-0 :warning: yetus 0m 2s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 :ok: mvndep 0m 16s Maven dependency ordering for branch
+1 :green_heart: mvninstall 3m 9s master passed
+1 :green_heart: compile 2m 12s master passed
+1 :green_heart: javadoc 2m 14s master passed
+1 :green_heart: shadedjars 5m 45s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 15s Maven dependency ordering for patch
-1 :x: mvninstall 0m 7s /patch-mvninstall-root.txt root in the patch failed.
-1 :x: compile 0m 8s /patch-compile-root.txt root in the patch failed.
-0 :warning: javac 0m 8s /patch-compile-root.txt root in the patch failed.
-0 :warning: javadoc 0m 15s /results-javadoc-javadoc-hbase-examples.txt hbase-examples generated 14 new + 0 unchanged - 0 fixed = 14 total (was 0)
-0 :warning: javadoc 1m 58s /results-javadoc-javadoc-root.txt root generated 14 new + 93 unchanged - 0 fixed = 107 total (was 93)
-1 :x: shadedjars 0m 7s patch has 9 errors when building our shaded downstream artifacts.
_ Other Tests _
-1 :x: unit 0m 8s /patch-unit-root.txt root in the patch failed.
18m 1s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28842
Optional Tests javac javadoc unit compile shadedjars
uname Linux bf5bdc37c146 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 0c60911fbc4737c98442a6880777d4bb69d98948
Default Java Eclipse Adoptium-17.0.11+9
shadedjars https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/2/artifact/yetus-jdk17-hadoop3-check/output/patch-shadedjars.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/2/testReport/
Max. process+thread count 178 (vs. ulimit of 30000)
modules C: hbase-examples . U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/2/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Sep 30 '24 19:09 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 39s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+0 :ok: mvndep 0m 16s Maven dependency ordering for branch
+1 :green_heart: mvninstall 3m 2s master passed
+1 :green_heart: compile 8m 12s master passed
+1 :green_heart: checkstyle 1m 8s master passed
+1 :green_heart: spotbugs 8m 15s master passed
+1 :green_heart: spotless 0m 43s branch has no errors when running spotless:check.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 15s Maven dependency ordering for patch
-1 :x: mvninstall 0m 8s /patch-mvninstall-root.txt root in the patch failed.
-1 :x: compile 0m 8s /patch-compile-root.txt root in the patch failed.
-0 :warning: javac 0m 8s /patch-compile-root.txt root in the patch failed.
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 1m 5s /results-checkstyle-root.txt root: The patch generated 14 new + 0 unchanged - 0 fixed = 14 total (was 0)
+1 :green_heart: xmllint 0m 0s No new issues.
-1 :x: spotbugs 0m 15s /patch-spotbugs-hbase-examples.txt hbase-examples in the patch failed.
-1 :x: spotbugs 0m 8s /patch-spotbugs-root.txt root in the patch failed.
-1 :x: hadoopcheck 0m 6s The patch causes 9 errors with Hadoop v3.3.6.
-1 :x: hadoopcheck 0m 13s The patch causes 9 errors with Hadoop v3.4.0.
-1 :x: spotless 0m 6s patch has 25 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
-1 :x: asflicense 0m 18s /results-asflicense.txt The patch generated 15 ASF License warnings.
26m 21s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28842
Optional Tests dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless xmllint
uname Linux c325af3cb9e1 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 0c60911fbc4737c98442a6880777d4bb69d98948
Default Java Eclipse Adoptium-17.0.11+9
hadoopcheck https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/2/artifact/yetus-general-check/output/patch-javac-3.3.6.txt
hadoopcheck https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/2/artifact/yetus-general-check/output/patch-javac-3.4.0.txt
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/2/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count 191 (vs. ulimit of 30000)
modules C: hbase-examples . U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/2/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3 xmllint=20913
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Sep 30 '24 19:09 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 44s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+0 :ok: mvndep 0m 17s Maven dependency ordering for branch
+1 :green_heart: mvninstall 4m 4s master passed
+1 :green_heart: compile 11m 4s master passed
+1 :green_heart: checkstyle 1m 15s master passed
+1 :green_heart: spotbugs 10m 13s master passed
+1 :green_heart: spotless 0m 55s branch has no errors when running spotless:check.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 16s Maven dependency ordering for patch
+1 :green_heart: mvninstall 3m 56s the patch passed
+1 :green_heart: compile 11m 19s the patch passed
-0 :warning: javac 11m 19s /results-compile-javac-root.txt root generated 20 new + 1215 unchanged - 0 fixed = 1235 total (was 1215)
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 1m 13s /results-checkstyle-root.txt root: The patch generated 11 new + 0 unchanged - 0 fixed = 11 total (was 0)
+1 :green_heart: xmllint 0m 0s No new issues.
+1 :green_heart: spotbugs 10m 53s the patch passed
+1 :green_heart: hadoopcheck 14m 12s Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
-1 :x: spotless 0m 49s patch has 21 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
+1 :green_heart: asflicense 0m 24s The patch does not generate ASF License warnings.
80m 5s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28842
Optional Tests dupname asflicense javac codespell detsecrets xmllint hadoopcheck spotless compile spotbugs checkstyle hbaseanti
uname Linux 1a292cfa6390 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / b59d35f975a3e9b0979d713e911705d76cff5df9
Default Java Eclipse Adoptium-17.0.11+9
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/3/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count 192 (vs. ulimit of 30000)
modules C: hbase-examples . U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/3/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3 xmllint=20913
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Sep 30 '24 22:09 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 49s Docker mode activated.
-0 :warning: yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 :ok: mvndep 0m 18s Maven dependency ordering for branch
+1 :green_heart: mvninstall 4m 17s master passed
+1 :green_heart: compile 3m 1s master passed
+1 :green_heart: javadoc 3m 30s master passed
+1 :green_heart: shadedjars 7m 27s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 14s Maven dependency ordering for patch
+1 :green_heart: mvninstall 3m 42s the patch passed
+1 :green_heart: compile 2m 46s the patch passed
+1 :green_heart: javac 2m 46s the patch passed
+1 :green_heart: javadoc 3m 11s the patch passed
+1 :green_heart: shadedjars 6m 38s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 :x: unit 336m 20s /patch-unit-root.txt root in the patch failed.
378m 25s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28842
Optional Tests javac javadoc unit shadedjars compile
uname Linux bfdd944ca1d4 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / b59d35f975a3e9b0979d713e911705d76cff5df9
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/3/testReport/
Max. process+thread count 8835 (vs. ulimit of 30000)
modules C: hbase-examples . U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/3/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Oct 01 '24 03:10 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 44s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+0 :ok: mvndep 0m 19s Maven dependency ordering for branch
+1 :green_heart: mvninstall 3m 37s master passed
+1 :green_heart: compile 11m 4s master passed
+1 :green_heart: checkstyle 1m 39s master passed
+1 :green_heart: spotbugs 12m 19s master passed
+1 :green_heart: spotless 0m 56s branch has no errors when running spotless:check.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 18s Maven dependency ordering for patch
+1 :green_heart: mvninstall 4m 16s the patch passed
+1 :green_heart: compile 14m 13s the patch passed
-0 :warning: javac 14m 13s /results-compile-javac-root.txt root generated 20 new + 1215 unchanged - 0 fixed = 1235 total (was 1215)
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 1m 51s /results-checkstyle-root.txt root: The patch generated 11 new + 0 unchanged - 0 fixed = 11 total (was 0)
+1 :green_heart: xmllint 0m 0s No new issues.
+1 :green_heart: spotbugs 12m 49s the patch passed
+1 :green_heart: hadoopcheck 15m 31s Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 :green_heart: spotless 1m 14s patch has no errors when running spotless:check.
_ Other Tests _
+1 :green_heart: asflicense 0m 34s The patch does not generate ASF License warnings.
91m 31s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28842
Optional Tests dupname asflicense javac codespell detsecrets xmllint hadoopcheck spotless compile spotbugs checkstyle hbaseanti
uname Linux cf4e53a7eeab 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 4b745957953590f167ac5c3252dcd56a701ae87a
Default Java Eclipse Adoptium-17.0.11+9
Max. process+thread count 193 (vs. ulimit of 30000)
modules C: hbase-examples . U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/4/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3 xmllint=20913
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Oct 01 '24 05:10 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 47s Docker mode activated.
-0 :warning: yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 :ok: mvndep 0m 19s Maven dependency ordering for branch
+1 :green_heart: mvninstall 3m 52s master passed
+1 :green_heart: compile 2m 47s master passed
+1 :green_heart: javadoc 3m 14s master passed
+1 :green_heart: shadedjars 6m 17s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 16s Maven dependency ordering for patch
+1 :green_heart: mvninstall 3m 38s the patch passed
+1 :green_heart: compile 2m 33s the patch passed
+1 :green_heart: javac 2m 33s the patch passed
+1 :green_heart: javadoc 2m 38s the patch passed
+1 :green_heart: shadedjars 6m 0s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 :green_heart: unit 360m 9s root in the patch passed.
399m 33s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28842
Optional Tests javac javadoc unit shadedjars compile
uname Linux 7aea89c7ed09 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 4b745957953590f167ac5c3252dcd56a701ae87a
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/4/testReport/
Max. process+thread count 8641 (vs. ulimit of 30000)
modules C: hbase-examples . U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/4/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Oct 01 '24 10:10 Apache-HBase

@eab148 if you want community eyes on this, I recommend that you send a note to the dev@ mailing list. I think most folks are not watching PRs or Jira issue creation.

ndimiduk avatar Oct 02 '24 15:10 ndimiduk

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 26s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 1s codespell was not available.
+0 :ok: detsecrets 0m 1s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+0 :ok: mvndep 0m 15s Maven dependency ordering for branch
+1 :green_heart: mvninstall 3m 7s master passed
+1 :green_heart: compile 7m 40s master passed
+1 :green_heart: checkstyle 1m 11s master passed
+1 :green_heart: spotbugs 7m 38s master passed
+1 :green_heart: spotless 0m 43s branch has no errors when running spotless:check.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 15s Maven dependency ordering for patch
+1 :green_heart: mvninstall 2m 58s the patch passed
+1 :green_heart: compile 7m 45s the patch passed
-0 :warning: javac 7m 45s /results-compile-javac-root.txt root generated 20 new + 1215 unchanged - 0 fixed = 1235 total (was 1215)
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 1m 8s /results-checkstyle-root.txt root: The patch generated 11 new + 0 unchanged - 0 fixed = 11 total (was 0)
+1 :green_heart: xmllint 0m 0s No new issues.
+1 :green_heart: spotbugs 7m 50s the patch passed
+1 :green_heart: hadoopcheck 11m 32s Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
-1 :x: spotless 0m 42s patch has 23 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
+1 :green_heart: asflicense 0m 20s The patch does not generate ASF License warnings.
60m 46s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28842
Optional Tests dupname asflicense javac codespell detsecrets xmllint hadoopcheck spotless compile spotbugs checkstyle hbaseanti
uname Linux 9739fe88470b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 87c47364daad8ac6ac1a1fa0ac593fca1839cd59
Default Java Eclipse Adoptium-17.0.11+9
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/5/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count 191 (vs. ulimit of 30000)
modules C: hbase-examples . U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/5/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3 xmllint=20913
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Oct 02 '24 16:10 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 39s Docker mode activated.
-0 :warning: yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 :ok: mvndep 0m 20s Maven dependency ordering for branch
+1 :green_heart: mvninstall 4m 13s master passed
+1 :green_heart: compile 2m 48s master passed
+1 :green_heart: javadoc 2m 41s master passed
+1 :green_heart: shadedjars 6m 57s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 16s Maven dependency ordering for patch
+1 :green_heart: mvninstall 3m 55s the patch passed
+1 :green_heart: compile 2m 34s the patch passed
+1 :green_heart: javac 2m 34s the patch passed
+1 :green_heart: javadoc 2m 37s the patch passed
+1 :green_heart: shadedjars 6m 20s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 :x: unit 271m 29s /patch-unit-root.txt root in the patch failed.
311m 19s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/5/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28842
Optional Tests javac javadoc unit shadedjars compile
uname Linux 0eaccd0b3bac 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 87c47364daad8ac6ac1a1fa0ac593fca1839cd59
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/5/testReport/
Max. process+thread count 5057 (vs. ulimit of 30000)
modules C: hbase-examples . U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/5/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Oct 02 '24 20:10 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 44s Docker mode activated.
-0 :warning: yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 3m 2s master passed
+1 :green_heart: compile 0m 21s master passed
+1 :green_heart: javadoc 0m 18s master passed
+1 :green_heart: shadedjars 5m 21s branch has no errors when building our shaded downstream artifacts.
-0 :warning: patch 5m 30s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 2m 56s the patch passed
+1 :green_heart: compile 0m 21s the patch passed
+1 :green_heart: javac 0m 21s the patch passed
+1 :green_heart: javadoc 0m 17s the patch passed
+1 :green_heart: shadedjars 5m 24s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 :green_heart: unit 1m 54s hbase-examples in the patch passed.
21m 51s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/6/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28842
Optional Tests javac javadoc unit shadedjars compile
uname Linux 63c1a00c1ddd 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / bc863abbb5752ce5a10654f62ebaba81dc7baad4
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/6/testReport/
Max. process+thread count 1461 (vs. ulimit of 30000)
modules C: hbase-examples U: hbase-examples
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/6/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Oct 03 '24 18:10 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 39s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+1 :green_heart: mvninstall 3m 4s master passed
+1 :green_heart: compile 0m 29s master passed
+1 :green_heart: checkstyle 0m 12s master passed
+1 :green_heart: spotbugs 0m 34s master passed
+1 :green_heart: spotless 0m 44s branch has no errors when running spotless:check.
-0 :warning: patch 0m 52s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 2m 57s the patch passed
+1 :green_heart: compile 0m 28s the patch passed
-0 :warning: javac 0m 28s /results-compile-javac-hbase-examples.txt hbase-examples generated 20 new + 15 unchanged - 0 fixed = 35 total (was 15)
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 0m 11s /results-checkstyle-hbase-examples.txt hbase-examples: The patch generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0)
+1 :green_heart: xmllint 0m 1s No new issues.
+1 :green_heart: spotbugs 0m 40s the patch passed
+1 :green_heart: hadoopcheck 10m 35s Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
-1 :x: spotless 0m 43s patch has 31 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
+1 :green_heart: asflicense 0m 10s The patch does not generate ASF License warnings.
28m 2s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/6/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28842
Optional Tests dupname asflicense javac codespell detsecrets xmllint hadoopcheck spotless compile spotbugs checkstyle hbaseanti
uname Linux 5e8f3b89303a 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / bc863abbb5752ce5a10654f62ebaba81dc7baad4
Default Java Eclipse Adoptium-17.0.11+9
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/6/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count 84 (vs. ulimit of 30000)
modules C: hbase-examples U: hbase-examples
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/6/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3 xmllint=20913
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Oct 03 '24 18:10 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 42s Docker mode activated.
-0 :warning: yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 2m 55s master passed
+1 :green_heart: compile 0m 22s master passed
+1 :green_heart: javadoc 0m 17s master passed
+1 :green_heart: shadedjars 5m 21s branch has no errors when building our shaded downstream artifacts.
-0 :warning: patch 5m 31s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 2m 58s the patch passed
+1 :green_heart: compile 0m 21s the patch passed
+1 :green_heart: javac 0m 21s the patch passed
+1 :green_heart: javadoc 0m 17s the patch passed
+1 :green_heart: shadedjars 5m 17s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 :green_heart: unit 1m 54s hbase-examples in the patch passed.
21m 15s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/7/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28842
Optional Tests javac javadoc unit shadedjars compile
uname Linux 52f4039ac260 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / a32c93444ac6c97971ed29b57b11c5e2f334253b
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/7/testReport/
Max. process+thread count 1445 (vs. ulimit of 30000)
modules C: hbase-examples U: hbase-examples
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/7/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Oct 04 '24 18:10 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 44s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+1 :green_heart: mvninstall 4m 8s master passed
+1 :green_heart: compile 0m 51s master passed
+1 :green_heart: checkstyle 0m 16s master passed
+1 :green_heart: spotbugs 0m 45s master passed
+1 :green_heart: spotless 0m 53s branch has no errors when running spotless:check.
-0 :warning: patch 1m 0s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 3m 15s the patch passed
+1 :green_heart: compile 0m 37s the patch passed
-0 :warning: javac 0m 37s /results-compile-javac-hbase-examples.txt hbase-examples generated 20 new + 15 unchanged - 0 fixed = 35 total (was 15)
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 0m 11s /results-checkstyle-hbase-examples.txt hbase-examples: The patch generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0)
+1 :green_heart: xmllint 0m 0s No new issues.
+1 :green_heart: spotbugs 0m 49s the patch passed
+1 :green_heart: hadoopcheck 10m 59s Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 :green_heart: spotless 0m 42s patch has no errors when running spotless:check.
_ Other Tests _
+1 :green_heart: asflicense 0m 10s The patch does not generate ASF License warnings.
30m 57s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/7/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28842
Optional Tests dupname asflicense javac codespell detsecrets xmllint hadoopcheck spotless compile spotbugs checkstyle hbaseanti
uname Linux 0c98d4980593 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / a32c93444ac6c97971ed29b57b11c5e2f334253b
Default Java Eclipse Adoptium-17.0.11+9
Max. process+thread count 82 (vs. ulimit of 30000)
modules C: hbase-examples U: hbase-examples
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/7/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3 xmllint=20913
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Oct 04 '24 18:10 Apache-HBase

@stoty @virajjasani is this something that Phoenix could use to make better query plans?

ndimiduk avatar Oct 14 '24 14:10 ndimiduk

Since i cannot leave a comment on the design doc, would you mind filling in some detail about how these statistics are queried and used? Thanks @eab148 !

ndimiduk avatar Oct 14 '24 15:10 ndimiduk

@stoty @virajjasani is this something that Phoenix could use to make better query plans?

Most of these statistics are not particularly relevant for Phoenix, as extreme row/cell sizes are less typical in Phoenix's relational model.

However, I think this same infrastructure could be extended for key distribution, cardinality, and other statistics that are relevant for query plan optimization. Those would probably need some kind of pluggalble interface where Phoenix can process the rowkeys as its own PK components as the data is collected.

Phoenix doesn't really do optimization based on table contents today, so there is a lot of room for improvement.

Unfortunately, my knowledge on optimization techniques is also lacking, so I cannot tell whether this infrastructure has any flaws that would later limit Phoenix's ability to use it.

stoty avatar Oct 14 '24 16:10 stoty

As for the data collection, one alternative could be using a special CF for the statistics, instead of a separate table. That way the statistics could be stored right with the table, and with the correct rowkey design (i.e. prepending the region's start key) it would also be local to the same region server.

Phoenix's local indexes use this trick.

If write performance is a problem, the rowkey for the statistics table could also be salted.

IUC we now have one cell per region per CF, so the the statistics write load might not even be an issue, it's a tradeoff between write and read performance, as usual.

stoty avatar Oct 14 '24 16:10 stoty

I think if we were to build this natively into hbase as a feature, it probably shouldn't be a coprocessor. It could be written within the core internals, which would afford us some extra power. Namely, we could store these statistics directly into the HFiles in the metadata. The query layer could handle aggregating across the HFiles of a region. We could provide coprocessor hooks to let users add their own data to the same per-HFile metadata.

bbeaudreault avatar Oct 14 '24 19:10 bbeaudreault

Since i cannot leave a comment on the design doc, would you mind filling in some detail about how these statistics are queried and used? Thanks @eab148 !

Querying row statistics:

We have an API that fetches and performs aggregations/filtering on our row statistics. These results are cached for a duration of time.

The available API queries include:

  • Get a given region's row statistics
  • Sample X row statistics for a given table/CF
  • Aggregate all of the row statistics across table/CF pairs
  • Fetch the "top N" row statistics for a given field and table/CF pair
    • In other words, we may provide LARGEST_ROW_NUM_BYTES and fetch the top N row statistics in order of RowStatistics::largestRowNumBytes (descending order)

As a reminder, the cells in our internal row statistics table have the following fields:

  • rowKey: full region name
  • family: 0
  • qualifier: 1 for majorCompaction row statistics, 0 for minor compaction row statistics
  • value: JSON blob of the RowStatistic object

Using row statistics:

At my day job, we've used the row statistics to

  1. Tune the block sizes for tables that service random read workloads, reducing disk I/O on relevant clusters
  2. Remove huge cells (>512 KiB) from our tables. Huge cells are ticking time bombs in HBase, as they cannot be cached in memory without admin intervention/memory configuration changes.
  3. Implement smarter compaction schedules, reducing the daily network data transfer cost of our HBase setup
Tune Block Sizes

Our block size tuning job halves the block size for each family of a table if

  • The table’s "typical row" can fit into the smaller block size. At my day job, clients usually query full rows, so we want all cells for a given row to be in the same block.
  • The cluster’s memory has space for the larger index size
  • The table serves mostly random read traffic.

Tuning the block sizes for sequential read workloads is more complicated as estimating the number of blocks a Scan will require is an ambiguous task when one considers filtered, partial, and full table Scans.

Remove Huge Cells
  1. Use our Row Statistics API to aggregate all of the row statistics for a given table.
  2. Use this response to find the number of cells that exceed the max cache size, which we have set to 512 KiB.
  3. Alert the relevant teams within our organization that they need to remove or break up these cells in their HBase tables.
Smarter Compaction Schedules
  1. Use our Row Statistics API to aggregate get the most recent row statistics for a given region.
  2. Estimate the amount of useful work that a major compaction accomplishes based on the number of StoreFiles in that region, the number of tombstones in that region, and the time since last compaction for that region

eab148 avatar Oct 17 '24 16:10 eab148

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 0s Docker mode activated.
-1 :x: docker 0m 6s Docker failed to build run-specific yetus/hbase:tp-3994}.
Subsystem Report/Notes
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28837
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/8/console
versions git=2.25.1
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Oct 17 '24 16:10 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 44s Docker mode activated.
-0 :warning: yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 3m 2s master passed
+1 :green_heart: compile 0m 21s master passed
+1 :green_heart: javadoc 0m 18s master passed
+1 :green_heart: shadedjars 5m 18s branch has no errors when building our shaded downstream artifacts.
-0 :warning: patch 5m 28s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
-1 :x: mvninstall 2m 58s /patch-mvninstall-root.txt root in the patch failed.
-1 :x: compile 0m 20s /patch-compile-hbase-examples.txt hbase-examples in the patch failed.
-0 :warning: javac 0m 20s /patch-compile-hbase-examples.txt hbase-examples in the patch failed.
+1 :green_heart: javadoc 0m 17s the patch passed
+1 :green_heart: shadedjars 5m 17s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 :x: unit 0m 19s /patch-unit-hbase-examples.txt hbase-examples in the patch failed.
19m 58s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/8/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28837
Optional Tests javac javadoc unit shadedjars compile
uname Linux 3e2ffe62e03f 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e4ac7d13a32da36ae7c6312280eb5036375e90ba
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/8/testReport/
Max. process+thread count 83 (vs. ulimit of 30000)
modules C: hbase-examples U: hbase-examples
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/8/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Oct 17 '24 17:10 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 30s Docker mode activated.
-0 :warning: yetus 0m 2s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 3m 4s master passed
+1 :green_heart: compile 0m 18s master passed
+1 :green_heart: javadoc 0m 16s master passed
+1 :green_heart: shadedjars 5m 31s branch has no errors when building our shaded downstream artifacts.
-0 :warning: patch 5m 39s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
-1 :x: mvninstall 2m 56s /patch-mvninstall-root.txt root in the patch failed.
-1 :x: compile 0m 17s /patch-compile-hbase-examples.txt hbase-examples in the patch failed.
-0 :warning: javac 0m 17s /patch-compile-hbase-examples.txt hbase-examples in the patch failed.
+1 :green_heart: javadoc 0m 15s the patch passed
+1 :green_heart: shadedjars 5m 34s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 :x: unit 0m 18s /patch-unit-hbase-examples.txt hbase-examples in the patch failed.
19m 56s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/9/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28837
Optional Tests javac javadoc unit shadedjars compile
uname Linux 57db93ec431f 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e56be7ea78170ca5701e84bc65c54dcab2aeae7f
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/9/testReport/
Max. process+thread count 84 (vs. ulimit of 30000)
modules C: hbase-examples U: hbase-examples
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/9/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Oct 21 '24 14:10 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 42s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+1 :green_heart: mvninstall 3m 45s master passed
+1 :green_heart: compile 0m 31s master passed
+1 :green_heart: checkstyle 0m 11s master passed
+1 :green_heart: spotbugs 0m 39s master passed
+1 :green_heart: spotless 0m 49s branch has no errors when running spotless:check.
-0 :warning: patch 0m 56s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
-1 :x: mvninstall 3m 32s /patch-mvninstall-root.txt root in the patch failed.
-1 :x: compile 0m 22s /patch-compile-hbase-examples.txt hbase-examples in the patch failed.
-0 :warning: javac 0m 22s /patch-compile-hbase-examples.txt hbase-examples in the patch failed.
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 0m 11s /results-checkstyle-hbase-examples.txt hbase-examples: The patch generated 8 new + 0 unchanged - 0 fixed = 8 total (was 0)
+1 :green_heart: xmllint 0m 0s No new issues.
-1 :x: spotbugs 0m 26s /patch-spotbugs-hbase-examples.txt hbase-examples in the patch failed.
-1 :x: hadoopcheck 3m 18s The patch causes 16 errors with Hadoop v3.3.6.
-1 :x: hadoopcheck 6m 55s The patch causes 16 errors with Hadoop v3.4.0.
-1 :x: spotless 0m 57s patch has 70 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
+1 :green_heart: asflicense 0m 9s The patch does not generate ASF License warnings.
20m 40s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/9/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28837
Optional Tests dupname asflicense javac codespell detsecrets xmllint hadoopcheck spotless compile spotbugs checkstyle hbaseanti
uname Linux ac7fa4b1ede4 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e56be7ea78170ca5701e84bc65c54dcab2aeae7f
Default Java Eclipse Adoptium-17.0.11+9
hadoopcheck https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/9/artifact/yetus-general-check/output/patch-javac-3.3.6.txt
hadoopcheck https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/9/artifact/yetus-general-check/output/patch-javac-3.4.0.txt
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/9/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count 88 (vs. ulimit of 30000)
modules C: hbase-examples U: hbase-examples
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/9/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3 xmllint=20913
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Oct 21 '24 14:10 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 27s Docker mode activated.
-0 :warning: yetus 0m 2s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 2m 59s master passed
+1 :green_heart: compile 0m 18s master passed
+1 :green_heart: javadoc 0m 16s master passed
+1 :green_heart: shadedjars 5m 31s branch has no errors when building our shaded downstream artifacts.
-0 :warning: patch 5m 38s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
-1 :x: mvninstall 2m 50s /patch-mvninstall-root.txt root in the patch failed.
-1 :x: compile 0m 16s /patch-compile-hbase-examples.txt hbase-examples in the patch failed.
-0 :warning: javac 0m 16s /patch-compile-hbase-examples.txt hbase-examples in the patch failed.
+1 :green_heart: javadoc 0m 15s the patch passed
+1 :green_heart: shadedjars 5m 26s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 :x: unit 0m 17s /patch-unit-hbase-examples.txt hbase-examples in the patch failed.
19m 32s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/10/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28837
Optional Tests javac javadoc unit shadedjars compile
uname Linux 48196e498632 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 8ad776e3032bc3b2dcfda6100ea34aed00aa567f
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/10/testReport/
Max. process+thread count 83 (vs. ulimit of 30000)
modules C: hbase-examples U: hbase-examples
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/10/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Oct 21 '24 20:10 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 1m 19s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+1 :green_heart: mvninstall 4m 15s master passed
+1 :green_heart: compile 0m 45s master passed
+1 :green_heart: checkstyle 0m 13s master passed
+1 :green_heart: spotbugs 0m 57s master passed
+1 :green_heart: spotless 1m 19s branch has no errors when running spotless:check.
-0 :warning: patch 1m 29s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
-1 :x: mvninstall 4m 35s /patch-mvninstall-root.txt root in the patch failed.
-1 :x: compile 0m 31s /patch-compile-hbase-examples.txt hbase-examples in the patch failed.
-0 :warning: javac 0m 31s /patch-compile-hbase-examples.txt hbase-examples in the patch failed.
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 0m 19s /results-checkstyle-hbase-examples.txt hbase-examples: The patch generated 8 new + 0 unchanged - 0 fixed = 8 total (was 0)
+1 :green_heart: xmllint 0m 1s No new issues.
-1 :x: spotbugs 0m 26s /patch-spotbugs-hbase-examples.txt hbase-examples in the patch failed.
-1 :x: hadoopcheck 4m 1s The patch causes 16 errors with Hadoop v3.3.6.
-1 :x: hadoopcheck 8m 39s The patch causes 16 errors with Hadoop v3.4.0.
-1 :x: spotless 0m 48s patch has 69 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
+1 :green_heart: asflicense 0m 11s The patch does not generate ASF License warnings.
26m 13s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/10/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28837
Optional Tests dupname asflicense javac codespell detsecrets xmllint hadoopcheck spotless compile spotbugs checkstyle hbaseanti
uname Linux 9cdd4d41e8ff 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 8ad776e3032bc3b2dcfda6100ea34aed00aa567f
Default Java Eclipse Adoptium-17.0.11+9
hadoopcheck https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/10/artifact/yetus-general-check/output/patch-javac-3.3.6.txt
hadoopcheck https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/10/artifact/yetus-general-check/output/patch-javac-3.4.0.txt
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/10/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count 83 (vs. ulimit of 30000)
modules C: hbase-examples U: hbase-examples
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/10/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3 xmllint=20913
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Oct 21 '24 20:10 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 49s Docker mode activated.
-0 :warning: yetus 0m 5s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 3m 49s master passed
+1 :green_heart: compile 0m 24s master passed
+1 :green_heart: javadoc 0m 22s master passed
+1 :green_heart: shadedjars 6m 10s branch has no errors when building our shaded downstream artifacts.
-0 :warning: patch 6m 18s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 3m 17s the patch passed
+1 :green_heart: compile 0m 23s the patch passed
+1 :green_heart: javac 0m 23s the patch passed
+1 :green_heart: javadoc 0m 18s the patch passed
+1 :green_heart: shadedjars 6m 35s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 :green_heart: unit 2m 16s hbase-examples in the patch passed.
25m 27s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/11/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28837
Optional Tests javac javadoc unit shadedjars compile
uname Linux 72cb85d8d8ce 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / ce36d5853ca6ed9d7c2d60894a84c67c69ba0f7b
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/11/testReport/
Max. process+thread count 1660 (vs. ulimit of 30000)
modules C: hbase-examples U: hbase-examples
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/11/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Nov 14 '24 22:11 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 29s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+1 :green_heart: mvninstall 3m 28s master passed
+1 :green_heart: compile 0m 25s master passed
+1 :green_heart: checkstyle 0m 10s master passed
+1 :green_heart: spotbugs 0m 29s master passed
+1 :green_heart: spotless 0m 43s branch has no errors when running spotless:check.
-0 :warning: patch 0m 49s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 2m 47s the patch passed
+1 :green_heart: compile 0m 24s the patch passed
-0 :warning: javac 0m 24s /results-compile-javac-hbase-examples.txt hbase-examples generated 21 new + 15 unchanged - 0 fixed = 36 total (was 15)
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 0m 9s /results-checkstyle-hbase-examples.txt hbase-examples: The patch generated 8 new + 0 unchanged - 0 fixed = 8 total (was 0)
+1 :green_heart: xmllint 0m 0s No new issues.
+1 :green_heart: spotbugs 0m 36s the patch passed
+1 :green_heart: hadoopcheck 10m 40s Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
-1 :x: spotless 0m 42s patch has 68 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
+1 :green_heart: asflicense 0m 10s The patch does not generate ASF License warnings.
27m 53s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/11/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28837
Optional Tests dupname asflicense javac codespell detsecrets xmllint hadoopcheck spotless compile spotbugs checkstyle hbaseanti
uname Linux 5a14bac2ae24 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / ce36d5853ca6ed9d7c2d60894a84c67c69ba0f7b
Default Java Eclipse Adoptium-17.0.11+9
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/11/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count 83 (vs. ulimit of 30000)
modules C: hbase-examples U: hbase-examples
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/11/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3 xmllint=20913
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Nov 14 '24 22:11 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 41s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+0 :ok: mvndep 0m 40s Maven dependency ordering for branch
+1 :green_heart: mvninstall 4m 5s master passed
+1 :green_heart: compile 10m 27s master passed
+1 :green_heart: checkstyle 1m 21s master passed
+1 :green_heart: spotbugs 10m 39s master passed
+1 :green_heart: spotless 1m 10s branch has no errors when running spotless:check.
-0 :warning: patch 1m 26s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 :ok: mvndep 0m 18s Maven dependency ordering for patch
-1 :x: mvninstall 3m 26s /patch-mvninstall-root.txt root in the patch failed.
-1 :x: compile 11m 1s /patch-compile-root.txt root in the patch failed.
-0 :warning: javac 11m 1s /patch-compile-root.txt root in the patch failed.
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
-0 :warning: checkstyle 2m 9s /results-checkstyle-root.txt root: The patch generated 10 new + 0 unchanged - 0 fixed = 10 total (was 0)
+1 :green_heart: xmllint 0m 1s No new issues.
-1 :x: spotbugs 0m 17s /patch-spotbugs-hbase-examples.txt hbase-examples in the patch failed.
-1 :x: spotbugs 10m 19s /patch-spotbugs-root.txt root in the patch failed.
-1 :x: hadoopcheck 4m 27s The patch causes 10 errors with Hadoop v3.3.6.
-1 :x: hadoopcheck 9m 6s The patch causes 10 errors with Hadoop v3.4.0.
-1 :x: spotless 0m 17s patch has 29 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
+1 :green_heart: asflicense 0m 37s The patch does not generate ASF License warnings.
68m 37s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/12/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6327
JIRA Issue HBASE-28837
Optional Tests dupname asflicense javac codespell detsecrets xmllint hadoopcheck spotless compile spotbugs checkstyle hbaseanti
uname Linux c351d872264b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 4a17edd51c902d8dab65e5ef1eaa28fde2f774a3
Default Java Eclipse Adoptium-17.0.11+9
hadoopcheck https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/12/artifact/yetus-general-check/output/patch-javac-3.3.6.txt
hadoopcheck https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/12/artifact/yetus-general-check/output/patch-javac-3.4.0.txt
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/12/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count 189 (vs. ulimit of 30000)
modules C: hbase-examples . U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6327/12/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3 xmllint=20913
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Nov 15 '24 00:11 Apache-HBase