HBASE-28672 Ensure large batches are not indefinitely blocked by quotas

[rmdmattingly]

https://issues.apache.org/jira/browse/HBASE-28672

At my day job we are trying to implement default quotas for a variety of access patterns. We began by introducing a default read IO limit per-user, per-machine — this has been very successful in reducing hotspots, even on clusters with thousands of distinct users.

While implementing a default writes/second throttle, I realized that doing so would put us in a precarious situation where large enough batches may never succeed. If your batch size is greater than your TimeLimiter's max throughput, then you will always fail in the quota estimation stage; this will stick the client in a backoff-and-retry loop until it times out. Meanwhile IO estimates are more optimistic, deliberately, which can let large requests do targeted oversubscription of an IO quota:

// assume 1 block required for reads. this is probably a low estimate, which is okay
readConsumed = numReads > 0 ? blockSizeBytes : 0;

This is okay because the Limiter's availability will go negative and force a longer backoff on subsequent requests. I believe this is preferable UX compared to a doomed throttling loop.

In this PR I introduce batch estimation that takes the limiter's max throughput into account, and I've added unit tests to validate the intended behavior.

@ndimiduk @hgromer @bozzkar

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Comment
+0 :ok:	reexec	0m 25s	Docker mode activated.
		_ Prechecks _
+1 :green_heart:	dupname	0m 0s	No case conflicting files found.
+1 :green_heart:	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 :green_heart:	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 4s	master passed
+1 :green_heart:	compile	2m 40s	master passed
+1 :green_heart:	checkstyle	0m 36s	master passed
+1 :green_heart:	spotless	0m 43s	branch has no errors when running spotless:check.
+1 :green_heart:	spotbugs	1m 26s	master passed
		_ Patch Compile Tests _
+1 :green_heart:	mvninstall	2m 44s	the patch passed
+1 :green_heart:	compile	2m 38s	the patch passed
+1 :green_heart:	javac	2m 38s	the patch passed
+1 :green_heart:	checkstyle	0m 35s	the patch passed
+1 :green_heart:	whitespace	0m 0s	The patch has no whitespace issues.
+1 :green_heart:	hadoopcheck	5m 5s	Patch does not cause any errors with Hadoop 3.3.6.
+1 :green_heart:	spotless	0m 40s	patch has no errors when running spotless:check.
+1 :green_heart:	spotbugs	1m 33s	the patch passed
		_ Other Tests _
+1 :green_heart:	asflicense	0m 10s	The patch does not generate ASF License warnings.
		28m 47s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/6003
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 7866b8ef6a04 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / ff1e52bfce
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	78 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Comment
+0 :ok:	reexec	0m 38s	Docker mode activated.
-0 :warning:	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 0s	master passed
+1 :green_heart:	compile	0m 55s	master passed
+1 :green_heart:	shadedjars	5m 12s	branch has no errors when building our shaded downstream artifacts.
+1 :green_heart:	javadoc	0m 29s	master passed
		_ Patch Compile Tests _
+1 :green_heart:	mvninstall	2m 48s	the patch passed
+1 :green_heart:	compile	0m 54s	the patch passed
+1 :green_heart:	javac	0m 54s	the patch passed
+1 :green_heart:	shadedjars	5m 8s	patch has no errors when building our shaded downstream artifacts.
+1 :green_heart:	javadoc	0m 28s	the patch passed
		_ Other Tests _
+1 :green_heart:	unit	220m 56s	hbase-server in the patch passed.
		245m 7s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/6003
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 5166b086b3f8 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / ff1e52bfce
Default Java	Eclipse Adoptium-17.0.10+7
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/1/testReport/
Max. process+thread count	4565 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Comment
+0 :ok:	reexec	0m 12s	Docker mode activated.
-0 :warning:	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 :green_heart:	mvninstall	2m 41s	master passed
+1 :green_heart:	compile	0m 43s	master passed
+1 :green_heart:	shadedjars	5m 11s	branch has no errors when building our shaded downstream artifacts.
+1 :green_heart:	javadoc	0m 26s	master passed
		_ Patch Compile Tests _
+1 :green_heart:	mvninstall	2m 25s	the patch passed
+1 :green_heart:	compile	0m 42s	the patch passed
+1 :green_heart:	javac	0m 42s	the patch passed
+1 :green_heart:	shadedjars	5m 7s	patch has no errors when building our shaded downstream artifacts.
+1 :green_heart:	javadoc	0m 25s	the patch passed
		_ Other Tests _
+1 :green_heart:	unit	232m 48s	hbase-server in the patch passed.
		255m 14s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/6003
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 615bf96519da 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / ff1e52bfce
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/1/testReport/
Max. process+thread count	5428 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Comment
+0 :ok:	reexec	0m 36s	Docker mode activated.
-0 :warning:	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 :green_heart:	mvninstall	2m 58s	master passed
+1 :green_heart:	compile	0m 50s	master passed
+1 :green_heart:	shadedjars	5m 20s	branch has no errors when building our shaded downstream artifacts.
+1 :green_heart:	javadoc	0m 28s	master passed
		_ Patch Compile Tests _
+1 :green_heart:	mvninstall	2m 48s	the patch passed
+1 :green_heart:	compile	0m 50s	the patch passed
+1 :green_heart:	javac	0m 50s	the patch passed
+1 :green_heart:	shadedjars	5m 15s	patch has no errors when building our shaded downstream artifacts.
+1 :green_heart:	javadoc	0m 26s	the patch passed
		_ Other Tests _
+1 :green_heart:	unit	241m 29s	hbase-server in the patch passed.
		265m 38s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/6003
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 0a3e4cc11b41 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / ff1e52bfce
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/1/testReport/
Max. process+thread count	4815 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[rmdmattingly]

That's a good question. Despite the existing behavior, I think the intention was always for this to be more of a soft limit — the TimedQuota proto actually calls the field soft_limit:

message TimedQuota {
  required TimeUnit time_unit = 1;
  optional uint64 soft_limit  = 2;
  optional float share = 3;
  optional QuotaScope scope  = 4 [default = MACHINE];
}

That said, intention is one thing and reality is another. I could also see people missing the hard limits if that's how they're using quotas.

I'd probably prefer to make the addition of a hard limit a separate effort, though, because my guess is that it will actually be quite a bit of work relative to its value. For example, coming up with good enough estimations of IO-heavy request load to earnestly guarantee any sort of "hard limit" is going to be tricky/impossible

[bbeaudreault]

Ideally for a multi request it would be able to execute whatever actions fit within the available quota. This maintains the hard limit but also gives a user more likelihood to eventually finish their request through multiple rpc calls. Similar to the handle of MultiActionResultTooLarge.

[rmdmattingly]

I agree with you, and considered the MARTL approach internally but didn't do a good job of documenting this publicly.

My pushback is not that we shouldn't add support for some sort of MultiActionBatchTooLarge exception, but rather that it would be a pretty significant lift and is, imo, not necessarily in the scope of this issue. It's always been possible to oversubscribe IO quotas, and this is advertised as a soft limit. Despite it being a soft limit, there's a bug here which throws certain request shapes into indefinite backoff/retry loops.

It's definitely righteous work to also think about improving the server's respect of the limit via request chunking, but I think that's sort of its own task and, given the difference in changeset required, I don't think a lack of MABTL exceptions should block our fixing of this bug

[bbeaudreault]

Would we have the same problem with write size?

[rmdmattingly]

Oh yeah, good point. I bet we would. I can look into a fix there too

[rmdmattingly]

Actually, there's no realistic problem with write size. At least, no more than read size. Both make very naive lowball estimates based on the number of reads/writes: https://github.com/apache/hbase/blob/1ffa127be8abf2b4ab0be1a6ebb148a539e7a73f/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/DefaultOperationQuota.java#L191-L203

It's theoretically possible for the estimate to exceed the max IO allowed though, so maybe it's worth putting limits in place for both the read and write size estimates here? What do you guys think?

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Comment
+0 :ok:	reexec	0m 25s	Docker mode activated.
		_ Prechecks _
+1 :green_heart:	dupname	0m 0s	No case conflicting files found.
+1 :green_heart:	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 :green_heart:	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 17s	master passed
+1 :green_heart:	compile	2m 39s	master passed
+1 :green_heart:	checkstyle	0m 35s	master passed
+1 :green_heart:	spotless	0m 42s	branch has no errors when running spotless:check.
+1 :green_heart:	spotbugs	1m 28s	master passed
		_ Patch Compile Tests _
+1 :green_heart:	mvninstall	2m 41s	the patch passed
+1 :green_heart:	compile	2m 36s	the patch passed
+1 :green_heart:	javac	2m 36s	the patch passed
+1 :green_heart:	checkstyle	0m 33s	the patch passed
+1 :green_heart:	whitespace	0m 0s	The patch has no whitespace issues.
+1 :green_heart:	hadoopcheck	5m 5s	Patch does not cause any errors with Hadoop 3.3.6.
+1 :green_heart:	spotless	0m 39s	patch has no errors when running spotless:check.
+1 :green_heart:	spotbugs	1m 31s	the patch passed
		_ Other Tests _
+1 :green_heart:	asflicense	0m 9s	The patch does not generate ASF License warnings.
		28m 41s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/6003
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 0a8f0edec554 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / f8aa3e2312
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Comment
+0 :ok:	reexec	0m 27s	Docker mode activated.
-0 :warning:	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 15s	master passed
+1 :green_heart:	compile	1m 3s	master passed
+1 :green_heart:	shadedjars	6m 52s	branch has no errors when building our shaded downstream artifacts.
+1 :green_heart:	javadoc	0m 32s	master passed
		_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 33s	the patch passed
+1 :green_heart:	compile	1m 5s	the patch passed
+1 :green_heart:	javac	1m 5s	the patch passed
+1 :green_heart:	shadedjars	6m 53s	patch has no errors when building our shaded downstream artifacts.
+1 :green_heart:	javadoc	0m 31s	the patch passed
		_ Other Tests _
+1 :green_heart:	unit	217m 3s	hbase-server in the patch passed.
		245m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/6003
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux f16ff78dc8e2 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / f8aa3e2312
Default Java	Eclipse Adoptium-17.0.10+7
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/2/testReport/
Max. process+thread count	4666 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Comment
+0 :ok:	reexec	0m 28s	Docker mode activated.
-0 :warning:	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 20s	master passed
+1 :green_heart:	compile	0m 48s	master passed
+1 :green_heart:	shadedjars	5m 48s	branch has no errors when building our shaded downstream artifacts.
+1 :green_heart:	javadoc	0m 26s	master passed
		_ Patch Compile Tests _
+1 :green_heart:	mvninstall	2m 57s	the patch passed
+1 :green_heart:	compile	0m 50s	the patch passed
+1 :green_heart:	javac	0m 50s	the patch passed
+1 :green_heart:	shadedjars	5m 44s	patch has no errors when building our shaded downstream artifacts.
+1 :green_heart:	javadoc	0m 23s	the patch passed
		_ Other Tests _
+1 :green_heart:	unit	222m 23s	hbase-server in the patch passed.
		247m 6s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/6003
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a1a63af91397 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / f8aa3e2312
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/2/testReport/
Max. process+thread count	5093 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Comment
+0 :ok:	reexec	0m 11s	Docker mode activated.
-0 :warning:	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 :green_heart:	mvninstall	2m 26s	master passed
+1 :green_heart:	compile	0m 43s	master passed
+1 :green_heart:	shadedjars	5m 6s	branch has no errors when building our shaded downstream artifacts.
+1 :green_heart:	javadoc	0m 24s	master passed
		_ Patch Compile Tests _
+1 :green_heart:	mvninstall	2m 25s	the patch passed
+1 :green_heart:	compile	0m 43s	the patch passed
+1 :green_heart:	javac	0m 43s	the patch passed
+1 :green_heart:	shadedjars	5m 11s	patch has no errors when building our shaded downstream artifacts.
+1 :green_heart:	javadoc	0m 25s	the patch passed
		_ Other Tests _
+1 :green_heart:	unit	232m 19s	hbase-server in the patch passed.
		254m 26s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/6003
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 97ae37989ba2 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / f8aa3e2312
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/2/testReport/
Max. process+thread count	4322 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[rmdmattingly]

I pushed an addition which ensures we don't block requests indefinitely for read, write, or request size estimates

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 44s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 0s		codespell was not available.
+0 :ok:	detsecrets	0m 0s		detect-secrets was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
+1 :green_heart:	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 24s		master passed
+1 :green_heart:	compile	3m 13s		master passed
+1 :green_heart:	checkstyle	0m 42s		master passed
+1 :green_heart:	spotbugs	1m 55s		master passed
+1 :green_heart:	spotless	1m 0s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 40s		the patch passed
+1 :green_heart:	compile	3m 28s		the patch passed
+1 :green_heart:	javac	3m 28s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
+1 :green_heart:	checkstyle	0m 43s		the patch passed
+1 :green_heart:	spotbugs	1m 52s		the patch passed
+1 :green_heart:	hadoopcheck	6m 2s		Patch does not cause any errors with Hadoop 3.3.6.
+1 :green_heart:	spotless	0m 58s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 :green_heart:	asflicense	0m 12s		The patch does not generate ASF License warnings.
		36m 4s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/6003
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux f1686cb007c0 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / c5cccfc5dae5b1d7c4737a8f723530baf9f49509
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	85 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/3/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 25s		Docker mode activated.
-0 :warning:	yetus	0m 2s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 19s		master passed
+1 :green_heart:	compile	0m 46s		master passed
+1 :green_heart:	javadoc	0m 22s		master passed
+1 :green_heart:	shadedjars	5m 32s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	2m 42s		the patch passed
+1 :green_heart:	compile	0m 45s		the patch passed
+1 :green_heart:	javac	0m 45s		the patch passed
+1 :green_heart:	javadoc	0m 22s		the patch passed
+1 :green_heart:	shadedjars	5m 32s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 :green_heart:	unit	215m 4s		hbase-server in the patch passed.
		239m 1s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/6003
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 623fe5179fa2 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / c5cccfc5dae5b1d7c4737a8f723530baf9f49509
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/3/testReport/
Max. process+thread count	4750 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 33s		Docker mode activated.
-0 :warning:	yetus	0m 2s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 42s		master passed
+1 :green_heart:	compile	1m 4s		master passed
+1 :green_heart:	javadoc	0m 39s		master passed
+1 :green_heart:	shadedjars	6m 29s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 27s		the patch passed
+1 :green_heart:	compile	1m 9s		the patch passed
+1 :green_heart:	javac	1m 9s		the patch passed
+1 :green_heart:	javadoc	0m 33s		the patch passed
+1 :green_heart:	shadedjars	6m 34s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 :green_heart:	unit	224m 13s		hbase-server in the patch passed.
		252m 49s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/6003
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 7d41a98d1eb5 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / c5cccfc5dae5b1d7c4737a8f723530baf9f49509
Default Java	Eclipse Adoptium-17.0.10+7
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/3/testReport/
Max. process+thread count	4713 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:broken_heart: -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 43s		Docker mode activated.
-0 :warning:	yetus	0m 2s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 :green_heart:	mvninstall	2m 54s		master passed
+1 :green_heart:	compile	0m 40s		master passed
+1 :green_heart:	javadoc	0m 25s		master passed
+1 :green_heart:	shadedjars	5m 43s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	2m 26s		the patch passed
+1 :green_heart:	compile	0m 40s		the patch passed
+1 :green_heart:	javac	0m 40s		the patch passed
+1 :green_heart:	javadoc	0m 23s		the patch passed
+1 :green_heart:	shadedjars	5m 43s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 :x:	unit	238m 10s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		262m 36s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/6003
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux eb7daa586d3f 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / c5cccfc5dae5b1d7c4737a8f723530baf9f49509
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/3/testReport/
Max. process+thread count	5283 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 35s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 0s		codespell was not available.
+0 :ok:	detsecrets	0m 0s		detect-secrets was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
+1 :green_heart:	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 53s		master passed
+1 :green_heart:	compile	3m 37s		master passed
+1 :green_heart:	checkstyle	0m 51s		master passed
+1 :green_heart:	spotbugs	1m 51s		master passed
+1 :green_heart:	spotless	0m 53s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	2m 51s		the patch passed
+1 :green_heart:	compile	2m 56s		the patch passed
+1 :green_heart:	javac	2m 56s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
+1 :green_heart:	checkstyle	0m 34s		the patch passed
+1 :green_heart:	spotbugs	1m 37s		the patch passed
+1 :green_heart:	hadoopcheck	5m 14s		Patch does not cause any errors with Hadoop 3.3.6.
+1 :green_heart:	spotless	0m 40s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 :green_heart:	asflicense	0m 9s		The patch does not generate ASF License warnings.
		32m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/6003
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 54b12a07b010 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 15b58667defe17d1ade6f177079da01d19b7e82b
Default Java	Eclipse Adoptium-17.0.10+7
Max. process+thread count	87 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/4/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 37s		Docker mode activated.
-0 :warning:	yetus	0m 2s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 :green_heart:	mvninstall	2m 59s		master passed
+1 :green_heart:	compile	0m 54s		master passed
+1 :green_heart:	javadoc	0m 29s		master passed
+1 :green_heart:	shadedjars	5m 11s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	2m 46s		the patch passed
+1 :green_heart:	compile	0m 54s		the patch passed
+1 :green_heart:	javac	0m 54s		the patch passed
+1 :green_heart:	javadoc	0m 28s		the patch passed
+1 :green_heart:	shadedjars	5m 13s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 :green_heart:	unit	220m 30s		hbase-server in the patch passed.
		244m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/6003
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 85783adace03 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 15b58667defe17d1ade6f177079da01d19b7e82b
Default Java	Eclipse Adoptium-17.0.10+7
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/4/testReport/
Max. process+thread count	4848 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6003/4/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.