hbase
hbase copied to clipboard
apache
HBASE-29658: Add RISC-V Vector (RVV) optimizations
Description
This PR introduces RISC-V Vector (RVV) optimizations to Apache HBase to significantly improve performance on RISC-V platforms. The optimizations target three critical areas:
- LZ4 Compression/Decompression - Vectorized hash computation and dictionary operations
- BloomFilter Operations - Parallel bit manipulation and batch processing
- Scan Query Operations - Vectorized byte comparisons and prefix matching
Key Features
- JNI integration with native C implementations using RVV intrinsics
- Conditional compilation ensuring compatibility with non-RISC-V platforms
- Runtime detection with graceful fallback mechanisms
- Comprehensive test coverage and performance benchmarks
- Backward compatibility maintained for existing deployments
Core Design Principles
- Conditional Compilation: Use
#if defined(__riscv) && defined(__riscv_vector)to ensure RVV code only compiles on supported platforms - Runtime Detection: Implement graceful fallback when RVV is unavailable
- Backward Compatibility: No impact on existing deployments or non-RISC-V platforms
- Performance Monitoring: Built-in metrics collection and performance analysis
JNI Integration Strategy
The implementation uses JNI to call native C functions that utilize RISC-V vector instructions for:
- Vectorized memory operations (memcmp, memcpy)
- Parallel hash computation for BloomFilter operations
- Batch processing for improved throughput
Closes #HBASE-29658
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 33s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | |
| _ master Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | |
| +1 :green_heart: | mvninstall | 3m 37s | master passed | |
| +1 :green_heart: | compile | 4m 28s | master passed | |
| +1 :green_heart: | checkstyle | 1m 0s | master passed | |
| +1 :green_heart: | spotbugs | 2m 27s | master passed | |
| +1 :green_heart: | spotless | 0m 53s | branch has no errors when running spotless:check. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | |
| -1 :x: | mvninstall | 0m 38s | /patch-mvninstall-root.txt | root in the patch failed. |
| -1 :x: | compile | 0m 15s | /patch-compile-hbase-common.txt | hbase-common in the patch failed. |
| -1 :x: | compile | 0m 39s | /patch-compile-hbase-server.txt | hbase-server in the patch failed. |
| -1 :x: | compile | 0m 13s | /patch-compile-hbase-compression_hbase-compression-lz4.txt | hbase-compression-lz4 in the patch failed. |
| -1 :x: | cc | 0m 15s | /patch-compile-hbase-common.txt | hbase-common in the patch failed. |
| -1 :x: | cc | 0m 39s | /patch-compile-hbase-server.txt | hbase-server in the patch failed. |
| -1 :x: | cc | 0m 13s | /patch-compile-hbase-compression_hbase-compression-lz4.txt | hbase-compression-lz4 in the patch failed. |
| -1 :x: | golang | 0m 15s | /patch-compile-hbase-common.txt | hbase-common in the patch failed. |
| -1 :x: | golang | 0m 39s | /patch-compile-hbase-server.txt | hbase-server in the patch failed. |
| -1 :x: | golang | 0m 13s | /patch-compile-hbase-compression_hbase-compression-lz4.txt | hbase-compression-lz4 in the patch failed. |
| -0 :warning: | javac | 0m 15s | /patch-compile-hbase-common.txt | hbase-common in the patch failed. |
| -0 :warning: | javac | 0m 39s | /patch-compile-hbase-server.txt | hbase-server in the patch failed. |
| -0 :warning: | javac | 0m 13s | /patch-compile-hbase-compression_hbase-compression-lz4.txt | hbase-compression-lz4 in the patch failed. |
| -0 :warning: | blanks | 0m 0s | /blanks-eol.txt | The patch has 48 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 0m 10s | /buildtool-patch-checkstyle-hbase-common.txt | The patch fails to run checkstyle in hbase-common |
| -0 :warning: | checkstyle | 0m 8s | /buildtool-patch-checkstyle-hbase-server.txt | The patch fails to run checkstyle in hbase-server |
| -0 :warning: | checkstyle | 0m 5s | /buildtool-patch-checkstyle-hbase-compression_hbase-compression-lz4.txt | The patch fails to run checkstyle in hbase-compression-lz4 |
| -1 :x: | spotbugs | 0m 12s | /patch-spotbugs-hbase-common.txt | hbase-common in the patch failed. |
| -1 :x: | spotbugs | 0m 30s | /patch-spotbugs-hbase-server.txt | hbase-server in the patch failed. |
| -1 :x: | spotbugs | 0m 12s | /patch-spotbugs-hbase-compression_hbase-compression-lz4.txt | hbase-compression-lz4 in the patch failed. |
| -1 :x: | hadoopcheck | 0m 43s | The patch causes 212 errors with Hadoop v3.3.6. | |
| -1 :x: | hadoopcheck | 1m 26s | The patch causes 212 errors with Hadoop v3.4.1. | |
| -1 :x: | spotless | 0m 7s | patch has 56 errors when running spotless:check, run spotless:apply to fix. | |
| _ Other Tests _ | ||||
| -1 :x: | asflicense | 0m 21s | /results-asflicense.txt | The patch generated 14 ASF License warnings. |
| 20m 16s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7388 |
| JIRA Issue | HBASE-29658 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless cc golang |
| uname | Linux 25242804c1c1 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 9396d729ac0c0c28e6ff30169b2f2d9a5d3ef57f |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| hadoopcheck | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/patch-javac-3.3.6.txt |
| hadoopcheck | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/patch-javac-3.4.1.txt |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/patch-spotless.txt |
| Max. process+thread count | 84 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server hbase-compression/hbase-compression-lz4 U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 56s | Docker mode activated. | |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck | |
| _ Prechecks _ | ||||
| _ master Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | |
| +1 :green_heart: | mvninstall | 5m 11s | master passed | |
| +1 :green_heart: | compile | 2m 37s | master passed | |
| +1 :green_heart: | javadoc | 1m 25s | master passed | |
| +1 :green_heart: | shadedjars | 9m 5s | branch has no errors when building our shaded downstream artifacts. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | |
| -1 :x: | mvninstall | 1m 9s | /patch-mvninstall-root.txt | root in the patch failed. |
| -1 :x: | compile | 0m 25s | /patch-compile-hbase-common.txt | hbase-common in the patch failed. |
| -1 :x: | compile | 0m 50s | /patch-compile-hbase-server.txt | hbase-server in the patch failed. |
| -1 :x: | compile | 0m 25s | /patch-compile-hbase-compression_hbase-compression-lz4.txt | hbase-compression-lz4 in the patch failed. |
| -0 :warning: | javac | 0m 25s | /patch-compile-hbase-common.txt | hbase-common in the patch failed. |
| -0 :warning: | javac | 0m 50s | /patch-compile-hbase-server.txt | hbase-server in the patch failed. |
| -0 :warning: | javac | 0m 25s | /patch-compile-hbase-compression_hbase-compression-lz4.txt | hbase-compression-lz4 in the patch failed. |
| -0 :warning: | javadoc | 0m 25s | /patch-javadoc-hbase-common.txt | hbase-common in the patch failed. |
| -0 :warning: | javadoc | 0m 29s | /patch-javadoc-hbase-server.txt | hbase-server in the patch failed. |
| -0 :warning: | javadoc | 0m 23s | /patch-javadoc-hbase-compression_hbase-compression-lz4.txt | hbase-compression-lz4 in the patch failed. |
| -1 :x: | shadedjars | 1m 30s | patch has 212 errors when building our shaded downstream artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 0m 22s | /patch-unit-hbase-common.txt | hbase-common in the patch failed. |
| -1 :x: | unit | 0m 44s | /patch-unit-hbase-server.txt | hbase-server in the patch failed. |
| -1 :x: | unit | 0m 22s | /patch-unit-hbase-compression_hbase-compression-lz4.txt | hbase-compression-lz4 in the patch failed. |
| 28m 42s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7388 |
| JIRA Issue | HBASE-29658 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 0a58c2361429 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 9396d729ac0c0c28e6ff30169b2f2d9a5d3ef57f |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| shadedjars | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-jdk17-hadoop3-check/output/patch-shadedjars.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/testReport/ |
| Max. process+thread count | 81 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server hbase-compression/hbase-compression-lz4 U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 15s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 1s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | |
| _ master Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | |
| +1 :green_heart: | mvninstall | 5m 38s | master passed | |
| +1 :green_heart: | compile | 15m 28s | master passed | |
| +1 :green_heart: | checkstyle | 2m 58s | master passed | |
| +1 :green_heart: | spotbugs | 17m 17s | master passed | |
| +1 :green_heart: | spotless | 1m 14s | branch has no errors when running spotless:check. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | |
| -1 :x: | mvninstall | 1m 1s | /patch-mvninstall-root.txt | root in the patch failed. |
| -1 :x: | compile | 1m 6s | /patch-compile-root.txt | root in the patch failed. |
| -1 :x: | cc | 1m 6s | /patch-compile-root.txt | root in the patch failed. |
| -1 :x: | golang | 1m 6s | /patch-compile-root.txt | root in the patch failed. |
| -0 :warning: | javac | 1m 6s | /patch-compile-root.txt | root in the patch failed. |
| -0 :warning: | blanks | 0m 0s | /blanks-eol.txt | The patch has 56 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 0m 17s | /buildtool-patch-checkstyle-root.txt | The patch fails to run checkstyle in root |
| -1 :x: | spotbugs | 0m 16s | /patch-spotbugs-hbase-common.txt | hbase-common in the patch failed. |
| -1 :x: | spotbugs | 0m 56s | /patch-spotbugs-hbase-server.txt | hbase-server in the patch failed. |
| -1 :x: | spotbugs | 3m 33s | /patch-spotbugs-root.txt | root in the patch failed. |
| -1 :x: | hadoopcheck | 1m 10s | The patch causes 16 errors with Hadoop v3.3.6. | |
| -1 :x: | hadoopcheck | 2m 16s | The patch causes 16 errors with Hadoop v3.4.1. | |
| -1 :x: | spotless | 0m 10s | patch has 63 errors when running spotless:check, run spotless:apply to fix. | |
| _ Other Tests _ | ||||
| -1 :x: | asflicense | 0m 45s | /results-asflicense.txt | The patch generated 14 ASF License warnings. |
| 56m 44s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7388 |
| JIRA Issue | HBASE-29658 |
| Optional Tests | dupname asflicense codespell detsecrets spotless javac spotbugs checkstyle compile hadoopcheck hbaseanti cc golang |
| uname | Linux 77e8e15e3d97 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / e8500d4f08940bb24861a27b7c2b60507d4d62dc |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| hadoopcheck | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/patch-javac-3.3.6.txt |
| hadoopcheck | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/patch-javac-3.4.1.txt |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-general-check/output/patch-spotless.txt |
| Max. process+thread count | 163 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server hbase-compression/hbase-compression-lz4 . U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 15s | Docker mode activated. | |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck | |
| _ Prechecks _ | ||||
| _ master Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for branch | |
| +1 :green_heart: | mvninstall | 5m 58s | master passed | |
| +1 :green_heart: | compile | 4m 27s | master passed | |
| +1 :green_heart: | javadoc | 5m 23s | master passed | |
| +1 :green_heart: | shadedjars | 10m 6s | branch has no errors when building our shaded downstream artifacts. | |
| _ Patch Compile Tests _ | ||||
| +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | |
| -1 :x: | mvninstall | 1m 5s | /patch-mvninstall-root.txt | root in the patch failed. |
| -1 :x: | compile | 0m 55s | /patch-compile-root.txt | root in the patch failed. |
| -0 :warning: | javac | 0m 55s | /patch-compile-root.txt | root in the patch failed. |
| -0 :warning: | javadoc | 0m 13s | /patch-javadoc-hbase-common.txt | hbase-common in the patch failed. |
| -0 :warning: | javadoc | 0m 40s | /results-javadoc-javadoc-hbase-server.txt | hbase-server generated 3 new + 63 unchanged - 0 fixed = 66 total (was 63) |
| -0 :warning: | javadoc | 0m 16s | /patch-javadoc-root.txt | root in the patch failed. |
| -1 :x: | shadedjars | 0m 9s | patch has 7 errors when building our shaded downstream artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 0m 53s | /patch-unit-root.txt | root in the patch failed. |
| 32m 54s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7388 |
| JIRA Issue | HBASE-29658 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 8497561e89d5 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / e8500d4f08940bb24861a27b7c2b60507d4d62dc |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| shadedjars | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/artifact/yetus-jdk17-hadoop3-check/output/patch-shadedjars.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/testReport/ |
| Max. process+thread count | 148 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-server hbase-compression/hbase-compression-lz4 . U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7388/1/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
The PR is really big...
Is it possible to contribute it with several smaller pieces so it is easier for us to review it?
Thanks.
