hbase icon indicating copy to clipboard operation
hbase copied to clipboard

HBASE-24637 - Reseek regression related to filter SKIP hinting

Open ramkrish86 opened this issue 4 years ago • 25 comments

This patch depends on what the scans or reads does with the blocks till the configured size. Just for understanding or initial review I have based this on the pread size (which is 4 * block size). This needs a better measure may be based on the number of blocks purely rather than the block size. But the idea is that if based on that config we find that the seekOrSkipToNextcol() has always landed in the immediate next block always go with next() rather than this seekToSkiptoNextcol(). In our tests it is revealed that the comparison that we do here in case of ensuring whether to seek or skipping is adding to the performance rather than the actual seek.

ramkrish86 avatar Nov 16 '20 14:11 ramkrish86

:broken_heart: -1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 1m 6s Docker mode activated.
-0 :warning: yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 3m 47s master passed
+1 :green_heart: compile 0m 57s master passed
+1 :green_heart: shadedjars 6m 35s branch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 38s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 3m 29s the patch passed
+1 :green_heart: compile 0m 56s the patch passed
+1 :green_heart: javac 0m 56s the patch passed
+1 :green_heart: shadedjars 6m 40s patch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 42s the patch passed
_ Other Tests _
-1 :x: unit 7m 28s hbase-server in the patch failed.
33m 40s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/2663
Optional Tests javac javadoc unit shadedjars compile
uname Linux d9f4ce0cdce5 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 4ee2270452
Default Java AdoptOpenJDK-1.8.0_232-b09
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/1/testReport/
Max. process+thread count 1025 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/1/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Nov 16 '20 15:11 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 1m 6s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 :green_heart: mvninstall 4m 8s master passed
+1 :green_heart: checkstyle 1m 14s master passed
+1 :green_heart: spotbugs 2m 11s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 3m 49s the patch passed
+1 :green_heart: checkstyle 1m 13s the patch passed
+1 :green_heart: whitespace 0m 0s The patch has no whitespace issues.
+1 :green_heart: hadoopcheck 18m 55s Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 :green_heart: spotbugs 2m 15s the patch passed
_ Other Tests _
+1 :green_heart: asflicense 0m 12s The patch does not generate ASF License warnings.
42m 49s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/2663
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname Linux f63fa45bf454 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 4ee2270452
Max. process+thread count 84 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/1/console
versions git=2.17.1 maven=3.6.3 spotbugs=3.1.12
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Nov 16 '20 15:11 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 1m 6s Docker mode activated.
-0 :warning: yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 4m 50s master passed
+1 :green_heart: compile 1m 13s master passed
+1 :green_heart: shadedjars 7m 20s branch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 43s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 4m 34s the patch passed
+1 :green_heart: compile 1m 11s the patch passed
+1 :green_heart: javac 1m 11s the patch passed
+1 :green_heart: shadedjars 7m 41s patch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 46s the patch passed
_ Other Tests _
-1 :x: unit 14m 22s hbase-server in the patch failed.
45m 9s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/2663
Optional Tests javac javadoc unit shadedjars compile
uname Linux b19642f334cf 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 4ee2270452
Default Java AdoptOpenJDK-11.0.6+10
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/1/testReport/
Max. process+thread count 671 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/1/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Nov 16 '20 15:11 Apache-HBase

Also I assume there is some kind of performance analysis that indicates an improvement? If not please provide one.

apurtell avatar Nov 16 '20 19:11 apurtell

:confetti_ball: +1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 1m 10s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 :green_heart: mvninstall 3m 56s master passed
+1 :green_heart: checkstyle 1m 13s master passed
+1 :green_heart: spotbugs 2m 5s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 3m 48s the patch passed
+1 :green_heart: checkstyle 1m 12s the patch passed
+1 :green_heart: whitespace 0m 0s The patch has no whitespace issues.
+1 :green_heart: hadoopcheck 18m 57s Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 :green_heart: spotbugs 2m 18s the patch passed
_ Other Tests _
+1 :green_heart: asflicense 0m 11s The patch does not generate ASF License warnings.
42m 35s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/2663
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname Linux b49c54dc5b1a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 8938b7a678
Max. process+thread count 84 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/2/console
versions git=2.17.1 maven=3.6.3 spotbugs=3.1.12
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Dec 01 '20 15:12 Apache-HBase

Thanks @apurtell - I will update the patch with test results and perf numbers.

ramkrish86 avatar Dec 01 '20 17:12 ramkrish86

:broken_heart: -1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 2m 33s Docker mode activated.
-0 :warning: yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 4m 46s master passed
+1 :green_heart: compile 1m 4s master passed
+1 :green_heart: shadedjars 7m 59s branch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 45s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 4m 18s the patch passed
+1 :green_heart: compile 1m 10s the patch passed
+1 :green_heart: javac 1m 10s the patch passed
+1 :green_heart: shadedjars 8m 17s patch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 41s the patch passed
_ Other Tests _
-1 :x: unit 153m 3s hbase-server in the patch failed.
186m 46s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/2663
Optional Tests javac javadoc unit shadedjars compile
uname Linux ea042ec9a344 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 8938b7a678
Default Java AdoptOpenJDK-1.8.0_232-b09
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/2/testReport/
Max. process+thread count 3866 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/2/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Dec 01 '20 17:12 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 1m 8s Docker mode activated.
-0 :warning: yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 4m 50s master passed
+1 :green_heart: compile 1m 13s master passed
+1 :green_heart: shadedjars 7m 20s branch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 45s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 4m 38s the patch passed
+1 :green_heart: compile 1m 13s the patch passed
+1 :green_heart: javac 1m 13s the patch passed
+1 :green_heart: shadedjars 7m 26s patch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 42s the patch passed
_ Other Tests _
-1 :x: unit 200m 49s hbase-server in the patch failed.
231m 57s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/2663
Optional Tests javac javadoc unit shadedjars compile
uname Linux d82f6b69e43a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 8938b7a678
Default Java AdoptOpenJDK-11.0.6+10
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/2/testReport/
Max. process+thread count 3520 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/2/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Dec 01 '20 18:12 Apache-HBase

  1. Data size = 2G - in a single region.
  2. All data major compacted and data in L1 cache
  3. ScanRange 10000 with 100 threads with FilterallFilter
  4. Each row with 80 cols each with 20 bytes value with all columns added.
  Avg latency (ms) Min latency (ms) Max latency (ms) TPS
Without patch 2561 2536 2568 39
With patch 1393 1380 1394 72

ramkrish86 avatar Dec 02 '20 08:12 ramkrish86

@infraio - Ping for review.

ramkrish86 avatar Dec 02 '20 08:12 ramkrish86

:broken_heart: -1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 1m 13s Docker mode activated.
-0 :warning: yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 3m 37s master passed
+1 :green_heart: compile 0m 58s master passed
+1 :green_heart: shadedjars 6m 57s branch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 37s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 3m 41s the patch passed
+1 :green_heart: compile 1m 0s the patch passed
+1 :green_heart: javac 1m 0s the patch passed
+1 :green_heart: shadedjars 6m 47s patch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 37s the patch passed
_ Other Tests _
-1 :x: unit 6m 53s hbase-server in the patch failed.
33m 38s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/2663
Optional Tests javac javadoc unit shadedjars compile
uname Linux fc9db0c8a7f7 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e40c626ae1
Default Java AdoptOpenJDK-1.8.0_232-b09
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/3/testReport/
Max. process+thread count 577 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/3/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Dec 02 '20 08:12 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 0m 34s Docker mode activated.
-0 :warning: yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 4m 24s master passed
+1 :green_heart: compile 1m 9s master passed
+1 :green_heart: shadedjars 7m 24s branch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 48s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 4m 29s the patch passed
+1 :green_heart: compile 1m 11s the patch passed
+1 :green_heart: javac 1m 11s the patch passed
+1 :green_heart: shadedjars 6m 57s patch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 41s the patch passed
_ Other Tests _
-1 :x: unit 7m 21s hbase-server in the patch failed.
36m 22s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/2663
Optional Tests javac javadoc unit shadedjars compile
uname Linux 32cef00ed556 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e40c626ae1
Default Java AdoptOpenJDK-11.0.6+10
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/3/testReport/
Max. process+thread count 709 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/3/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Dec 02 '20 08:12 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 0m 33s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 :green_heart: mvninstall 3m 46s master passed
+1 :green_heart: checkstyle 1m 6s master passed
+1 :green_heart: spotbugs 2m 2s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 4m 50s the patch passed
+1 :green_heart: checkstyle 1m 5s the patch passed
+1 :green_heart: whitespace 0m 0s The patch has no whitespace issues.
+1 :green_heart: hadoopcheck 17m 27s Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 :green_heart: spotbugs 2m 10s the patch passed
_ Other Tests _
+1 :green_heart: asflicense 0m 15s The patch does not generate ASF License warnings.
40m 25s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/2663
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname Linux 8ab3a0db6c88 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e40c626ae1
Max. process+thread count 94 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/3/console
versions git=2.17.1 maven=3.6.3 spotbugs=3.1.12
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Dec 02 '20 08:12 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 1m 14s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 :green_heart: mvninstall 4m 13s master passed
+1 :green_heart: checkstyle 1m 16s master passed
+1 :green_heart: spotbugs 2m 24s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 3m 55s the patch passed
+1 :green_heart: checkstyle 1m 13s the patch passed
+1 :green_heart: whitespace 0m 0s The patch has no whitespace issues.
+1 :green_heart: hadoopcheck 19m 42s Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 :green_heart: spotbugs 2m 51s the patch passed
_ Other Tests _
+1 :green_heart: asflicense 0m 15s The patch does not generate ASF License warnings.
46m 23s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/2663
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname Linux b479f2c1030f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e40c626ae1
Max. process+thread count 84 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/4/console
versions git=2.17.1 maven=3.6.3 spotbugs=3.1.12
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Dec 02 '20 10:12 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 1m 11s Docker mode activated.
-0 :warning: yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 3m 27s master passed
+1 :green_heart: compile 0m 57s master passed
+1 :green_heart: shadedjars 6m 45s branch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 37s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 3m 35s the patch passed
+1 :green_heart: compile 0m 58s the patch passed
+1 :green_heart: javac 0m 58s the patch passed
+1 :green_heart: shadedjars 6m 39s patch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 38s the patch passed
_ Other Tests _
+1 :green_heart: unit 146m 11s hbase-server in the patch passed.
173m 9s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/2663
Optional Tests javac javadoc unit shadedjars compile
uname Linux 049a4ff82204 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e40c626ae1
Default Java AdoptOpenJDK-1.8.0_232-b09
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/4/testReport/
Max. process+thread count 4579 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/4/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Dec 02 '20 12:12 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 1m 48s Docker mode activated.
-0 :warning: yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 5m 6s master passed
+1 :green_heart: compile 1m 18s master passed
+1 :green_heart: shadedjars 7m 40s branch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 47s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 4m 48s the patch passed
+1 :green_heart: compile 1m 18s the patch passed
+1 :green_heart: javac 1m 18s the patch passed
+1 :green_heart: shadedjars 7m 39s patch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 45s the patch passed
_ Other Tests _
-1 :x: unit 199m 21s hbase-server in the patch failed.
232m 27s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/2663
Optional Tests javac javadoc unit shadedjars compile
uname Linux 70456e96469d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e40c626ae1
Default Java AdoptOpenJDK-11.0.6+10
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/4/testReport/
Max. process+thread count 3460 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/4/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Dec 02 '20 13:12 Apache-HBase

@infraio I have updated the PR with a new config 'hbase.switchto.next.bytes.read' . It defaults to preadMaxBytes though. also if you configure this value to be < 0 then we don't honour this optimization.

ramkrish86 avatar Dec 10 '20 07:12 ramkrish86

:confetti_ball: +1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 0m 38s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 :green_heart: mvninstall 4m 47s master passed
+1 :green_heart: checkstyle 1m 35s master passed
+1 :green_heart: spotbugs 2m 48s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 4m 18s the patch passed
+1 :green_heart: checkstyle 1m 16s the patch passed
+1 :green_heart: whitespace 0m 0s The patch has no whitespace issues.
+1 :green_heart: hadoopcheck 22m 35s Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 :green_heart: spotbugs 2m 10s the patch passed
_ Other Tests _
+1 :green_heart: asflicense 0m 11s The patch does not generate ASF License warnings.
47m 45s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/2663
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname Linux a4006de01c24 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 7a532f8328
Max. process+thread count 94 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/5/console
versions git=2.17.1 maven=3.6.3 spotbugs=3.1.12
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Dec 10 '20 08:12 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 1m 24s Docker mode activated.
-0 :warning: yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 3m 48s master passed
+1 :green_heart: compile 0m 54s master passed
+1 :green_heart: shadedjars 6m 29s branch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 39s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 3m 33s the patch passed
+1 :green_heart: compile 0m 56s the patch passed
+1 :green_heart: javac 0m 56s the patch passed
+1 :green_heart: shadedjars 6m 29s patch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 37s the patch passed
_ Other Tests _
+1 :green_heart: unit 144m 3s hbase-server in the patch passed.
171m 8s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/2663
Optional Tests javac javadoc unit shadedjars compile
uname Linux bdf219ca8781 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 7a532f8328
Default Java AdoptOpenJDK-1.8.0_232-b09
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/5/testReport/
Max. process+thread count 4439 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/5/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Dec 10 '20 10:12 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Comment
+0 :ok: reexec 1m 9s Docker mode activated.
-0 :warning: yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 :green_heart: mvninstall 5m 4s master passed
+1 :green_heart: compile 1m 36s master passed
+1 :green_heart: shadedjars 8m 16s branch has no errors when building our shaded downstream artifacts.
+1 :green_heart: javadoc 0m 47s master passed
_ Patch Compile Tests _
+1 :green_heart: mvninstall 4m 55s the patch passed
+1 :green_heart: compile 1m 25s the patch passed
+1 :green_heart: javac 1m 25s the patch passed
+1 :green_heart: shadedjars 7m 50s patch has no errors when building our shaded downstream artifacts.
-0 :warning: javadoc 0m 45s hbase-server generated 1 new + 81 unchanged - 0 fixed = 82 total (was 81)
_ Other Tests _
+1 :green_heart: unit 219m 56s hbase-server in the patch passed.
253m 53s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/2663
Optional Tests javac javadoc unit shadedjars compile
uname Linux 8ce47c47d0a0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 7a532f8328
Default Java AdoptOpenJDK-11.0.6+10
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/5/artifact/yetus-jdk11-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/5/testReport/
Max. process+thread count 2758 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2663/5/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Dec 10 '20 12:12 Apache-HBase

@infraio - Can you take a look at the new commit? @Apache9 - Do you have some time to take a look at this?

ramkrish86 avatar Dec 14 '20 10:12 ramkrish86

Will take a look this week.

Can go ahead if you have other approvers and I haven't replied in time.

Thanks.

Apache9 avatar Dec 14 '20 13:12 Apache9

@ramkrish86 Any update here sir?

saintstack avatar Mar 16 '21 00:03 saintstack

Any update here?

bbeaudreault avatar Sep 03 '21 11:09 bbeaudreault

I have updated the PR with a new config 'hbase.switchto.next.bytes.read' .

No new config. Repeating my earlier comment. What user is actually going to understand and set this? Figure it out without user intervention. Please don't merge until this is addressed.

apurtell avatar Sep 08 '21 18:09 apurtell