hbase icon indicating copy to clipboard operation
hbase copied to clipboard

HBASE-28637 asyncwal should attempt to recover lease if close fails (#5962)

Open jojochuang opened this issue 1 year ago • 9 comments

Signed-off-by: Duo Zhang [email protected] (cherry picked from commit 0e8cfdb507723729a2cade50ea364149eec3466c) (cherry picked from commit b88129617ca1bb06948bd07a972c8a4f2121c02c)

Conflicts: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java

Change-Id: Ia704c1b36b99be345d11a70ac2452dea094e9575

jojochuang avatar Jun 27 '24 20:06 jojochuang

Changes made to address code conflicts: (1) moved the recoverLease call from AbstractFSWAL to AsyncFSWAL. (2) For FSHLog, move the logic inside closeWriter.

jojochuang avatar Jun 27 '24 20:06 jojochuang

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 42s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
_ branch-2 Compile Tests _
+1 :green_heart: mvninstall 3m 14s branch-2 passed
+1 :green_heart: compile 2m 47s branch-2 passed
+1 :green_heart: checkstyle 0m 37s branch-2 passed
+1 :green_heart: spotbugs 1m 26s branch-2 passed
-1 :x: spotless 0m 45s branch has 1 errors when running spotless:check, run spotless:apply to fix.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 2m 50s the patch passed
+1 :green_heart: compile 2m 47s the patch passed
+1 :green_heart: javac 2m 47s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 32s the patch passed
+1 :green_heart: spotbugs 1m 33s the patch passed
+1 :green_heart: hadoopcheck 9m 45s Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6.
-1 :x: spotless 0m 37s patch has 31 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
+1 :green_heart: asflicense 0m 11s The patch does not generate ASF License warnings.
29m 33s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6033
Optional Tests dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname Linux 24e6a771fd47 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 015f1d96ea5558fe70a04a5e5b32f9731310b1d6
Default Java Eclipse Adoptium-11.0.17+8
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/1/artifact/yetus-general-check/output/branch-spotless.txt
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/1/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count 79 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/1/console
versions git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Jun 27 '24 20:06 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 18s Docker mode activated.
-0 :warning: yetus 0m 5s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 :green_heart: mvninstall 3m 2s branch-2 passed
+1 :green_heart: compile 1m 0s branch-2 passed
+1 :green_heart: javadoc 0m 39s branch-2 passed
+1 :green_heart: shadedjars 6m 29s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 3m 14s the patch passed
+1 :green_heart: compile 1m 16s the patch passed
+1 :green_heart: javac 1m 17s the patch passed
+1 :green_heart: javadoc 0m 48s the patch passed
+1 :green_heart: shadedjars 6m 22s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 :x: unit 216m 19s /patch-unit-hbase-server.txt hbase-server in the patch failed.
243m 56s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6033
Optional Tests javac javadoc unit compile shadedjars
uname Linux ce6337a71711 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 015f1d96ea5558fe70a04a5e5b32f9731310b1d6
Default Java Eclipse Adoptium-17.0.10+7
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/1/testReport/
Max. process+thread count 4732 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/1/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Jun 28 '24 00:06 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 52s Docker mode activated.
-0 :warning: yetus 0m 5s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 :green_heart: mvninstall 2m 53s branch-2 passed
+1 :green_heart: compile 0m 46s branch-2 passed
+1 :green_heart: javadoc 0m 30s branch-2 passed
+1 :green_heart: shadedjars 5m 57s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 2m 43s the patch passed
+1 :green_heart: compile 0m 49s the patch passed
+1 :green_heart: javac 0m 49s the patch passed
+1 :green_heart: javadoc 0m 29s the patch passed
+1 :green_heart: shadedjars 5m 53s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 :green_heart: unit 244m 7s hbase-server in the patch passed.
269m 29s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6033
Optional Tests javac javadoc unit compile shadedjars
uname Linux 5c104100c13a 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 015f1d96ea5558fe70a04a5e5b32f9731310b1d6
Default Java Temurin-1.8.0_352-b08
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/1/testReport/
Max. process+thread count 4233 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/1/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Jun 28 '24 00:06 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 58s Docker mode activated.
-0 :warning: yetus 0m 6s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 :green_heart: mvninstall 3m 26s branch-2 passed
+1 :green_heart: compile 0m 54s branch-2 passed
+1 :green_heart: javadoc 0m 33s branch-2 passed
+1 :green_heart: shadedjars 7m 20s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 3m 40s the patch passed
+1 :green_heart: compile 1m 6s the patch passed
+1 :green_heart: javac 1m 6s the patch passed
+1 :green_heart: javadoc 0m 35s the patch passed
+1 :green_heart: shadedjars 7m 32s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 :green_heart: unit 260m 26s hbase-server in the patch passed.
291m 13s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6033
Optional Tests javac javadoc unit compile shadedjars
uname Linux 8a498a4a6056 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 015f1d96ea5558fe70a04a5e5b32f9731310b1d6
Default Java Eclipse Adoptium-11.0.17+8
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/1/testReport/
Max. process+thread count 4257 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/1/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Jun 28 '24 00:06 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 42s Docker mode activated.
_ Prechecks _
+1 :green_heart: dupname 0m 0s No case conflicting files found.
+0 :ok: codespell 0m 0s codespell was not available.
+0 :ok: detsecrets 0m 0s detect-secrets was not available.
+1 :green_heart: @author 0m 0s The patch does not contain any @author tags.
+1 :green_heart: hbaseanti 0m 0s Patch does not have any anti-patterns.
_ branch-2 Compile Tests _
+1 :green_heart: mvninstall 2m 44s branch-2 passed
+1 :green_heart: compile 2m 44s branch-2 passed
+1 :green_heart: checkstyle 0m 34s branch-2 passed
+1 :green_heart: spotbugs 1m 25s branch-2 passed
+1 :green_heart: spotless 0m 42s branch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 2m 46s the patch passed
+1 :green_heart: compile 2m 43s the patch passed
+1 :green_heart: javac 2m 43s the patch passed
+1 :green_heart: blanks 0m 0s The patch has no blanks issues.
+1 :green_heart: checkstyle 0m 35s the patch passed
+1 :green_heart: spotbugs 1m 33s the patch passed
+1 :green_heart: hadoopcheck 9m 40s Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6.
+1 :green_heart: spotless 0m 41s patch has no errors when running spotless:check.
_ Other Tests _
+1 :green_heart: asflicense 0m 8s The patch does not generate ASF License warnings.
28m 48s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6033
Optional Tests dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname Linux 53fb52b65c90 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / ef211850e7532b777ac67042cd751ddf4ad7be54
Default Java Eclipse Adoptium-11.0.17+8
Max. process+thread count 79 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/2/console
versions git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Jun 28 '24 01:06 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 20s Docker mode activated.
-0 :warning: yetus 0m 5s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 :green_heart: mvninstall 2m 57s branch-2 passed
+1 :green_heart: compile 0m 55s branch-2 passed
+1 :green_heart: javadoc 0m 29s branch-2 passed
+1 :green_heart: shadedjars 5m 21s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 2m 46s the patch passed
+1 :green_heart: compile 0m 54s the patch passed
+1 :green_heart: javac 0m 54s the patch passed
+1 :green_heart: javadoc 0m 29s the patch passed
+1 :green_heart: shadedjars 5m 21s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 :green_heart: unit 209m 1s hbase-server in the patch passed.
233m 4s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6033
Optional Tests javac javadoc unit compile shadedjars
uname Linux b767c1750ecd 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / ef211850e7532b777ac67042cd751ddf4ad7be54
Default Java Eclipse Adoptium-17.0.10+7
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/2/testReport/
Max. process+thread count 4752 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/2/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Jun 28 '24 04:06 Apache-HBase

:confetti_ball: +1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 59s Docker mode activated.
-0 :warning: yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 :green_heart: mvninstall 2m 57s branch-2 passed
+1 :green_heart: compile 0m 39s branch-2 passed
+1 :green_heart: javadoc 0m 24s branch-2 passed
+1 :green_heart: shadedjars 6m 20s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 3m 15s the patch passed
+1 :green_heart: compile 0m 54s the patch passed
+1 :green_heart: javac 0m 54s the patch passed
+1 :green_heart: javadoc 0m 32s the patch passed
+1 :green_heart: shadedjars 7m 12s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 :green_heart: unit 220m 15s hbase-server in the patch passed.
247m 30s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6033
Optional Tests javac javadoc unit compile shadedjars
uname Linux 210eeb12fcd2 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / ef211850e7532b777ac67042cd751ddf4ad7be54
Default Java Temurin-1.8.0_352-b08
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/2/testReport/
Max. process+thread count 4277 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/2/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Jun 28 '24 05:06 Apache-HBase

:broken_heart: -1 overall

Vote Subsystem Runtime Logfile Comment
+0 :ok: reexec 0m 52s Docker mode activated.
-0 :warning: yetus 0m 6s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 :green_heart: mvninstall 2m 53s branch-2 passed
+1 :green_heart: compile 0m 53s branch-2 passed
+1 :green_heart: javadoc 0m 29s branch-2 passed
+1 :green_heart: shadedjars 7m 7s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 :green_heart: mvninstall 3m 39s the patch passed
+1 :green_heart: compile 0m 57s the patch passed
+1 :green_heart: javac 0m 57s the patch passed
+1 :green_heart: javadoc 0m 30s the patch passed
+1 :green_heart: shadedjars 6m 57s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 :x: unit 268m 19s /patch-unit-hbase-server.txt hbase-server in the patch failed.
297m 6s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR https://github.com/apache/hbase/pull/6033
Optional Tests javac javadoc unit compile shadedjars
uname Linux b1d43cbf49c6 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / ef211850e7532b777ac67042cd751ddf4ad7be54
Default Java Eclipse Adoptium-11.0.17+8
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/2/testReport/
Max. process+thread count 4168 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6033/2/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase avatar Jun 28 '24 05:06 Apache-HBase

@Apache9 can i have your review again? Due to code refactors, the changes for branch-2 is quite different.

jojochuang avatar Jul 08 '24 19:07 jojochuang

Thanks @Apache9

jojochuang avatar Jul 10 '24 18:07 jojochuang