hbase
hbase copied to clipboard
apache
HBASE-27034 NegativeArraySizeException was encountered during compaction
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 38s | Docker mode activated. |
| _ Prechecks _ | |||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| _ master Compile Tests _ | |||
| +1 :green_heart: | mvninstall | 3m 56s | master passed |
| +1 :green_heart: | compile | 2m 10s | master passed |
| +1 :green_heart: | checkstyle | 0m 31s | master passed |
| +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 1m 14s | master passed |
| _ Patch Compile Tests _ | |||
| +1 :green_heart: | mvninstall | 2m 3s | the patch passed |
| +1 :green_heart: | compile | 2m 9s | the patch passed |
| +1 :green_heart: | javac | 2m 9s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 27s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| -1 :x: | hadoopcheck | 8m 20s | The patch causes 10 errors with Hadoop v3.3.1. |
| +1 :green_heart: | spotless | 0m 37s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 1m 19s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| 25m 42s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4430/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4430 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux 8a4cfb2d7c6a 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 369adcdf35 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| hadoopcheck | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4430/1/artifact/yetus-general-check/output/patch-javac-3.3.1.txt |
| Max. process+thread count | 65 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4430/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 37s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +1 :green_heart: | mvninstall | 4m 15s | master passed |
| +1 :green_heart: | compile | 0m 41s | master passed |
| +1 :green_heart: | shadedjars | 3m 51s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 30s | master passed |
| _ Patch Compile Tests _ | |||
| +1 :green_heart: | mvninstall | 2m 24s | the patch passed |
| +1 :green_heart: | compile | 0m 41s | the patch passed |
| +1 :green_heart: | javac | 0m 41s | the patch passed |
| +1 :green_heart: | shadedjars | 3m 44s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 23s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 175m 47s | hbase-server in the patch passed. |
| 194m 32s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4430/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4430 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux bf41c565f8a0 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 369adcdf35 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4430/1/testReport/ |
| Max. process+thread count | 2966 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4430/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| +0 :ok: | reexec | 0m 51s | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
| _ Prechecks _ | |||
| _ master Compile Tests _ | |||
| +1 :green_heart: | mvninstall | 4m 28s | master passed |
| +1 :green_heart: | compile | 0m 41s | master passed |
| +1 :green_heart: | shadedjars | 4m 24s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 27s | master passed |
| _ Patch Compile Tests _ | |||
| +1 :green_heart: | mvninstall | 2m 31s | the patch passed |
| +1 :green_heart: | compile | 0m 41s | the patch passed |
| +1 :green_heart: | javac | 0m 41s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 20s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 24s | the patch passed |
| _ Other Tests _ | |||
| +1 :green_heart: | unit | 191m 12s | hbase-server in the patch passed. |
| 211m 38s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4430/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4430 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 4d77107fd846 5.4.0-1068-aws #72~18.04.1-Ubuntu SMP Thu Mar 3 08:49:49 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 369adcdf35 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4430/1/testReport/ |
| Max. process+thread count | 2946 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4430/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
Can you describe when this can cause the problem? I don't know, I add some code to print lastCellOfPreviousBlock and i found it's value was modified, but i don't know where it was modified, We have a table for storing logs and this problem happens many times every day but other table not. Other people seem to have encountered this problem. https://developer.aliyun.com/ask/387299
The change here will impact performance, so we need more detailed analysis on what is the root cause, and then decide how to better fix it.
The change here will impact performance, so we need more detailed analysis on what is the root cause, and then decide how to better fix it. HBASE-16372 has explain why references to previous cell in read path will lead to incorrect results. So method beforeShipped of writer was called when bytesWrittenProgressForShippedCall > shippedCallSizeLimit , But this trigger condition looks incorrect
The change here will impact performance, so we need more detailed analysis on what is the root cause, and then decide how to better fix it. HBASE-16372 has explain why references to previous cell in read path will lead to incorrect results. So method beforeShipped of writer was called when bytesWrittenProgressForShippedCall > shippedCallSizeLimit , But this trigger condition looks incorrect
Mind explaining why it is incorrect?
Mind explaining why it is incorrect?
For example, if the number of files to compact is 10 and blocksize is 64k, beforeShipped was called once when every 640k was writeed. How does it make sure the the block that containing lastCellOfPreviousBlock will not be evicted in the meantime? I can't understand. More details in org.apache.hadoop.hbase.regionserver.compactions.Compactor#performCompaction,