apache
HBASE-29726 Allow updating time limit on ScannerContext
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 40s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | |
| _ master Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 4m 56s | master passed | |
| +1 :green_heart: | compile | 4m 28s | master passed | |
| +1 :green_heart: | checkstyle | 1m 6s | master passed | |
| +1 :green_heart: | spotbugs | 2m 3s | master passed | |
| +1 :green_heart: | spotless | 0m 58s | branch has no errors when running spotless:check. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 4m 10s | the patch passed | |
| +1 :green_heart: | compile | 4m 2s | the patch passed | |
| +1 :green_heart: | javac | 4m 2s | the patch passed | |
| -0 :warning: | blanks | 0m 0s | /blanks-eol.txt | The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 1m 8s | the patch passed | |
| +1 :green_heart: | spotbugs | 2m 3s | the patch passed | |
| +1 :green_heart: | hadoopcheck | 17m 44s | Patch does not cause any errors with Hadoop 3.3.6 3.4.1. | |
| -1 :x: | spotless | 0m 47s | patch has 25 errors when running spotless:check, run spotless:apply to fix. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | |
| 54m 36s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7475/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7475 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 33057db80b0a 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 55fee71c0a394e3130537ef8e73725d00b3ea98a |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7475/1/artifact/yetus-general-check/output/patch-spotless.txt |
| Max. process+thread count | 84 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7475/1/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 42s | Docker mode activated. | |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck | |
| _ Prechecks _ | ||||
| _ master Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 4m 58s | master passed | |
| +1 :green_heart: | compile | 1m 36s | master passed | |
| +1 :green_heart: | javadoc | 0m 37s | master passed | |
| +1 :green_heart: | shadedjars | 7m 21s | branch has no errors when building our shaded downstream artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 4m 11s | the patch passed | |
| +1 :green_heart: | compile | 1m 14s | the patch passed | |
| +1 :green_heart: | javac | 1m 14s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 39s | the patch passed | |
| +1 :green_heart: | shadedjars | 6m 51s | patch has no errors when building our shaded downstream artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 220m 39s | hbase-server in the patch passed. | |
| 253m 48s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7475/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/7475 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux d6926035a30b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 55fee71c0a394e3130537ef8e73725d00b3ea98a |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7475/1/testReport/ |
| Max. process+thread count | 3760 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7475/1/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
Could you please describe more on why and how Phoenix plan to control the time limit?
Thanks.
Phoenix currently has a home-grown solution (from back when Phoenix was primarily targeting HBase 1.x, which did not have the heartbeat feature ) for the heartbeat problem which uses a magic "Dummy" Cell to indicate heartbeat and cursor information, and has introduced its own configuration option for that time limit.
This works fine for Hbase 1 and 2, but in Hbase 3 it interferes with the heartbeat logic in the HBase client, causing it to skip rows.
The goal is the same as in HBase: allow long-running scans without having to set unrealistically long timeouts.
In a green-field implementation the HBase defaults would probably be good enough for Phoenix, but for backward compatibility reasons I'd like to be able to retain the ability to set the scan.next() time limit based on the existing parameter passed by the client in a scan attribute .
The current logic is centered in https://github.com/apache/phoenix/blob/master/phoenix-core-server/src/main/java/org/apache/phoenix/coprocessor/PagingRegionScanner.java
Here, instead of using PagingFilter, Phoenix could modify the ScannerContext time limit to the same value, and rely on the HBase functionality.
We can do that today, but we'd have to read all values from the existing ScannerContext and then create a new one, which is quite a lot of cpu cycles:
- we'd have to get the limits and progress ftom the original ScannerContext,
- then create a new one with the new time limit,
- then after the call copy the progress back to the original ScannerContext