hadoop
hadoop copied to clipboard
Published
20 hours ago •
apache
apache
HDFS-16689. Standby NameNode crashes when transitioning to Active with in-progress tailer
Description of PR
Standby NameNode may crash when transitioning to Active with a in-progress tailer if there are some abnormal JNs. And the crashed error message as blew:
Caused by: java.lang.IllegalStateException: Cannot start writing at txid X when there is a stream available for read: ByteStringEditLog[X, Y], ByteStringEditLog[X, 0]
at org.apache.hadoop.hdfs.server.namenode.FSEditLog.openForWrite(FSEditLog.java:344)
at org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync.openForWrite(FSEditLogAsync.java:113)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startActiveServices(FSNamesystem.java:1423)
at org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.startActiveServices(NameNode.java:2132)
... 36 more
After tracing and found there is a critical bug in EditlogTailer#catchupDuringFailover() with DFS_HA_TAILEDITS_INPROGRESS_KEY=true.
catchupDuringFailover() may cannot replay all edits from JournalNodes with onlyDurableTxns=true if there are some abnormal JournalNodes, because maybe JournalNodes will return a empty response caused maxAllowedTxns=0 in QuorumJournalManager#selectRpcInputStreams.
Reproduce method, suppose:
- There are 2 namenode, namely NN0 and NN1, and the status of echo namenode is Active, Standby respectively. And there are 3 JournalNodes, namely JN0, JN1 and JN2.
- NN0 try to sync 3 edits to JNs with started txid 3, but only successfully synced them to JN1 and JN3. And JN0 is abnormal, such as GC, bad network or restarted.
- NN1's lastAppliedTxId is 2, and at the moment, we are trying failover active from NN0 to NN1.
- NN1 only got two responses from JN0 and JN1 when it try to selecting inputStreams with
fromTxnId=3andonlyDurableTxns=true, and the count txid of response is 0, 3 respectively. JN2 is abnormal, such as GC, bad network or restarted. - NN1 will cannot replay any Edits with
fromTxnId=3from JournalNodes because themaxAllowedTxnsis 0.
So I think Standby NameNode should catchupDuringFailover() with onlyDurableTxns=false , so that it can replay all missed edits from JournalNode.
@ferhui @xkrogen Master, this PR uses getEditLogManifest() to fix this problem. Please help me review it, thanks.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 35s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 38m 4s | trunk passed | |
| +1 :green_heart: | compile | 1m 43s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | compile | 1m 38s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 26s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 50s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 26s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 48s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 41s | trunk passed | |
| +1 :green_heart: | shadedclient | 25m 2s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 1m 30s | the patch passed | |
| +1 :green_heart: | compile | 1m 40s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javac | 1m 40s | the patch passed | |
| +1 :green_heart: | compile | 1m 24s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 1m 24s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 0m 59s | /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs-project/hadoop-hdfs: The patch generated 10 new + 113 unchanged - 0 fixed = 123 total (was 113) |
| +1 :green_heart: | mvnsite | 1m 27s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 58s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 32s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 29s | the patch passed | |
| +1 :green_heart: | shadedclient | 23m 7s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 241m 15s | hadoop-hdfs in the patch passed. | |
| +1 :green_heart: | asflicense | 1m 14s | The patch does not generate ASF License warnings. | |
| 353m 59s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4744 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 9cd64b6f59b4 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0ca77ca0e9976dc0aef231841ac0bcd632e1a63f |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/1/testReport/ |
| Max. process+thread count | 3757 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 38s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 1s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 38m 16s | trunk passed | |
| +1 :green_heart: | compile | 1m 45s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | compile | 1m 39s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 23s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 48s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 25s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 48s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 44s | trunk passed | |
| +1 :green_heart: | shadedclient | 23m 2s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 1m 26s | the patch passed | |
| +1 :green_heart: | compile | 1m 27s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javac | 1m 27s | the patch passed | |
| +1 :green_heart: | compile | 1m 18s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 1m 18s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 1m 1s | the patch passed | |
| +1 :green_heart: | mvnsite | 1m 29s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 57s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 32s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 25s | the patch passed | |
| +1 :green_heart: | shadedclient | 22m 53s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 240m 23s | hadoop-hdfs in the patch passed. | |
| +1 :green_heart: | asflicense | 1m 14s | The patch does not generate ASF License warnings. | |
| 350m 50s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4744 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 790ea73b89ab 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8c4d268898c28a97c51bd7a35dfb0ca8a6b236b6 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/2/testReport/ |
| Max. process+thread count | 3453 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
It looks like the current diff is disallowing in-progress edits during catchupDuringFailover(). But this isn't right, we do want in-progress edits, since those can be durable and have been ack'ed to clients. Right?
I'm also a little confused because your last comment says you solved the issue via getEditLogManifest(), but I don't see that method called at all in the current diff.
@xkrogen Thanks for your review and comment. Sorry for the late replay.
Before catchupDuringFailover, whatever active crashed or successfully changed to standby, the last segment in majority journalnode should be finalized. My idea is catchupDuringFailover just ignore selectRpcInputStreams and fail back to use selectStreamingInputStreams with getEditLogManifest.
Unfortunately, catchupDuringFailover can only disable in-progress to ignore selectRpcInputStreams, because inProgressTailingEnabled in QuorumJournalManager is unchangeable.
The key code is as follows：
@Override
public void selectInputStreams(Collection<EditLogInputStream> streams,
long fromTxnId, boolean inProgressOk,
boolean onlyDurableTxns) throws IOException {
// Here, catchupDuringFailover should ignore this if branch and fail back to selectStreamingInputStreams
if (inProgressOk && inProgressTailingEnabled) {
LOG.debug("Tailing edits starting from txn ID {} via RPC mechanism", fromTxnId);
try {
Collection<EditLogInputStream> rpcStreams = new ArrayList<>();
selectRpcInputStreams(rpcStreams, fromTxnId, onlyDurableTxns);
streams.addAll(rpcStreams);
return;
} catch (IOException ioe) {
LOG.warn("Encountered exception while tailing edits >= " + fromTxnId +
" via RPC; falling back to streaming.", ioe);
}
}
selectStreamingInputStreams(streams, fromTxnId, inProgressOk,
onlyDurableTxns);
}
Before
catchupDuringFailover, whatever active crashed or successfully changed to standby, the last segment in majority journalnode should be finalized.
If the active crashed, then the segment won't be finalized, right?
Actually my recollection is a little hazy here. I guess the new active (former standby) will finalize the old segment, since the old active (which crashed) didn't do it. If the finalization happens before catchupDuringFailover, then I guess this approach might work.
The same processing idea has also appeared in HDFS-14806. Here
This is different. Bootstrap standby doesn't need to get all transactions, it's more like a best-effort to load most of the transactions. Later, when the standby transitions to active, then it will call catchupDuringFailover to load the remainder.
Though generally I agree that the idea is similar. Perhaps we should add a way for callers of QuorumJournalManager to indicate that they want to use the streaming mechanism, as opposed to RPC. Disabling in-progress edits achieves this, but is too strong (note that the streaming mechanism can also load in-progress edits).
If the active crashed, then the segment won't be finalized, right?
If the active crashed, during standby starting active services, the standby will recover unclosed streams via recoverUnclosedStreams. So before catchupDuringFailover, the last segment should always closed.
Disabling in-progress edits achieves this, but is too strong (note that the streaming mechanism can also load in-progress edits).
Nice suggestion, I will fix this patch like this way. Thanks, Master!
@xkrogen Master, I have update this patch via enable or disable inProgressTailing in QuorumJournalManager, please help me review it. If you have any good ideas, I'd be happy to modify this patch as desired.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 40s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 1s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 38m 22s | trunk passed | |
| +1 :green_heart: | compile | 1m 44s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | compile | 1m 37s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 26s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 47s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 25s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 47s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 39s | trunk passed | |
| +1 :green_heart: | shadedclient | 23m 14s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 1m 22s | the patch passed | |
| +1 :green_heart: | compile | 1m 27s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javac | 1m 27s | the patch passed | |
| +1 :green_heart: | compile | 1m 21s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 1m 21s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 1m 3s | the patch passed | |
| +1 :green_heart: | mvnsite | 1m 26s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 58s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 27s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 24s | the patch passed | |
| +1 :green_heart: | shadedclient | 22m 39s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 240m 21s | hadoop-hdfs in the patch passed. | |
| +1 :green_heart: | asflicense | 1m 13s | The patch does not generate ASF License warnings. | |
| 350m 27s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4744 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux bd3e3c5509b8 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a5ade31e6f1dc9674d288de910363b1b35f4520c |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/3/testReport/ |
| Max. process+thread count | 3206 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
If the active crashed, during standby starting active services, the standby will recover unclosed streams via
recoverUnclosedStreams. So beforecatchupDuringFailover, the last segment should always closed.
I don't think this will always be true. In recoverUnclosedStreams, if the finalization fails, it will just ignore it and assume that it will be handled later (by Journal#startLogSegment(), which will automatically close an old stream when you try to open a new one).
https://github.com/apache/hadoop/blob/63db1a85e376c2266afdc62b9590e40acc98429c/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java#L1660-L1670
So if we want to disable in-progress edits during catchupDuringFailover, then we have to modify this to retry/abort when the old segments cannot be closed.
But I don't think this is the right approach. I think you might have also understood what I was saying in my last comment here:
Perhaps we should add a way for callers of QuorumJournalManager to indicate that they want to use the streaming mechanism, as opposed to RPC. Disabling in-progress edits achieves this, but is too strong (note that the streaming mechanism can also load in-progress edits).
Looking at the current diff, I don't see a reason to add LogsPurgeable#enableInProgressTailing(). We already have the inProgressOk parameter of selectInputStreams(), so why do we need a different mechanism to adjust whether we use in-progress edits?
What I was trying to say is that what we really want to do, both in this Jira and in HDFS-14806, is disable the RPC mechanism and use the streaming mechanism. In HDFS-14806, to do this, we turned off in-progress edits, which was okay. But in this Jira, it's not okay -- we don't want the RPC mechanism, but we still need in-progress edits.
So basically I am saying we need something like the inProgressOk parameter, but for RPC vs. streaming.
I am thinking we can add a new parameter to LogsPurgeable#selectInputStreams() like preferBulkReads. RPC vs streaming is an implementation detail of QJM, but "bulk reads" could be general. If this parameter is true, then QJM can prefer the streaming mechanism (other JournalManagers can ignore it). Callers like BootstrapStandby#checkLogsAvailableForRead, BackupImage#tryConvergeJournalSpool, FSImage#loadFSImage, and of course EditLogTailer#catchupDuringFailover() can pass preferBulkReads = true.
Thinking more on this, maybe it does make sense to just modify startActiveServices to retry recoverUnclosedStreams until it actually succeeds. If there is something preventing the new active from communicating with the JNs, or something preventing the JNs from finalizing the old segment, then the NN will eventually fail to become active regardless.
@xkrogen Thanks.
I think you might have also understood what I was saying in my last comment
Yes, I got it.
I am thinking we can add a new parameter to LogsPurgeable#selectInputStreams() like preferBulkReads.
This is a good idea, I will fix this patch like this.
In recoverUnclosedStreams, if the finalization fails, it will just ignore it and assume that it will be handled later (by Journal#startLogSegment(), which will automatically close an old stream when you try to open a new one).
In recoverUnclosedStreams, if the finalization fails, it will just ignore it
Sorry, I didn't notice this. But I think it's crazy.
assume that it will be handled later (by Journal#startLogSegment(), which will automatically close an old stream when you try to open a new one).
I'm sorry, I just find this comment, but didn't find related code to finalize the previous inProgress segment. Can you share the related code? Thanks.
If there is something preventing the new active from communicating with the JNs, or something preventing the JNs from finalizing the old segment, then the NN will eventually fail to become active regardless.
Yes, I agree. Standby should crash or fail to become active if it cannot finalize the old segment. About this case, How about fix it in a new PR?
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 37s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 1s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 7 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 40m 8s | trunk passed | |
| +1 :green_heart: | compile | 1m 40s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | compile | 1m 38s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 26s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 46s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 23s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 48s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 43s | trunk passed | |
| +1 :green_heart: | shadedclient | 23m 31s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 1m 27s | the patch passed | |
| +1 :green_heart: | compile | 1m 28s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javac | 1m 28s | the patch passed | |
| +1 :green_heart: | compile | 1m 22s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 1m 22s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 1m 2s | the patch passed | |
| +1 :green_heart: | mvnsite | 1m 30s | the patch passed | |
| +1 :green_heart: | javadoc | 1m 4s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 30s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 24s | the patch passed | |
| +1 :green_heart: | shadedclient | 23m 11s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 249m 15s | hadoop-hdfs in the patch passed. | |
| +1 :green_heart: | asflicense | 1m 13s | The patch does not generate ASF License warnings. | |
| 361m 59s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4744 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 74f55473e3da 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 342e712086a910debbabc7ee799f999bd1557caa |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/4/testReport/ |
| Max. process+thread count | 3311 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
@abhishekkarigar Thanks for your attention to this issue.
@xkrogen and I will solve this problem as soon as possible.
@xkrogen Sir, please review the latest patch. Thanks
I'm sorry, I just find this comment, but didn't find related code to finalize the previous inProgress segment. Can you share the related code? Thanks.
I'm referring to this: https://github.com/apache/hadoop/blob/62c86eaa0e539a4307ca794e0fcd502a77ebceb8/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/Journal.java#L574-L583
But a little more digging made me realize that I don't think what I described will actually happen, since in FSEditLog#openForWrite() before calling startLogSegment() we first check that there are no active streams:
https://github.com/apache/hadoop/blob/63db1a85e376c2266afdc62b9590e40acc98429c/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java#L338-L347
So it will actually throw an exception, rather than finalizing the old segment as I said previously. But this is after catchupDuringFailver(), so to make your original proposal (disable in-progress edits) work properly, we still need to modify recoverUnclosedStreams() to throw an error when it fails instead of just swallowing the exception.
I briefly looked at the other usages of recoverUnclosedStreams() and I don't really see any reason why we would want to swallow the exception ... The TODO comment there is also from 2012, 10 years old now :)
So are we agreed that the best way forward is to modify recoverUnclosedStreams() to throw exception on failure, then we can use inProgressOk = false to solve this problem as you originally proposed?
if (curSegment != null) {
LOG.warn("Client is requesting a new log segment " + txid +
" though we are already writing " + curSegment + ". " +
"Aborting the current segment in order to begin the new one." +
" ; journal id: " + journalId);
// The writer may have lost a connection to us and is now
// re-connecting after the connection came back.
// We should abort our own old segment.
abortCurSegment();
}
The abortCurSegment() just aborts the current segment, but not finalize the current inProgress segment, so may result in two inProgress segment files on disk.
So are we agreed that the best way forward is to modify recoverUnclosedStreams() to throw exception on failure, then we can use inProgressOk = false to solve this problem as you originally proposed?
Yes, I totally agree with this and I will modify this patch with this idea.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 40s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 39m 19s | trunk passed | |
| +1 :green_heart: | compile | 1m 33s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | compile | 1m 29s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 15s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 38s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 24s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 45s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 35s | trunk passed | |
| +1 :green_heart: | shadedclient | 22m 44s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 1m 21s | the patch passed | |
| +1 :green_heart: | compile | 1m 25s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javac | 1m 25s | the patch passed | |
| +1 :green_heart: | compile | 1m 17s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 1m 17s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 1m 3s | the patch passed | |
| +1 :green_heart: | mvnsite | 1m 29s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 57s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 28s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 27s | the patch passed | |
| +1 :green_heart: | shadedclient | 23m 34s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 252m 8s | /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 4s | The patch does not generate ASF License warnings. | |
| 362m 37s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.hdfs.qjournal.TestNNWithQJM |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4744 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 4188a87c263a 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7d8b49790968f54bbc943fb3697624a9007d2126 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/5/testReport/ |
| Max. process+thread count | 2937 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 38s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 38m 16s | trunk passed | |
| +1 :green_heart: | compile | 1m 36s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | compile | 1m 27s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 23s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 43s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 15s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 39s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 28s | trunk passed | |
| +1 :green_heart: | shadedclient | 22m 45s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 1m 21s | the patch passed | |
| +1 :green_heart: | compile | 1m 24s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javac | 1m 24s | the patch passed | |
| +1 :green_heart: | compile | 1m 18s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 1m 18s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 0m 57s | /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 253 unchanged - 0 fixed = 254 total (was 253) |
| +1 :green_heart: | mvnsite | 1m 23s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 57s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 28s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 24s | the patch passed | |
| +1 :green_heart: | shadedclient | 22m 44s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 239m 12s | hadoop-hdfs in the patch passed. | |
| +1 :green_heart: | asflicense | 1m 1s | The patch does not generate ASF License warnings. | |
| 347m 32s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4744 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e86124f1fc66 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7f72f67ecf733794ff2df3797c16d03a29c146a9 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/6/testReport/ |
| Max. process+thread count | 3377 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 35s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 1s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 38m 55s | trunk passed | |
| +1 :green_heart: | compile | 1m 32s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | compile | 1m 25s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 14s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 44s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 16s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 44s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 48s | trunk passed | |
| +1 :green_heart: | shadedclient | 22m 34s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 1m 24s | the patch passed | |
| +1 :green_heart: | compile | 1m 23s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javac | 1m 23s | the patch passed | |
| +1 :green_heart: | compile | 1m 14s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 1m 14s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 1m 1s | /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 261 unchanged - 1 fixed = 263 total (was 262) |
| +1 :green_heart: | mvnsite | 1m 27s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 57s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 29s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 37s | the patch passed | |
| +1 :green_heart: | shadedclient | 22m 41s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 257m 29s | hadoop-hdfs in the patch passed. | |
| +1 :green_heart: | asflicense | 1m 5s | The patch does not generate ASF License warnings. | |
| 366m 9s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/7/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4744 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 8acc78f52b6e 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d45f42f4a538eaa90b2296722b46c781f6c4aa17 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/7/testReport/ |
| Max. process+thread count | 3264 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 41s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 38m 54s | trunk passed | |
| +1 :green_heart: | compile | 1m 43s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | compile | 1m 29s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 23s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 45s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 16s | trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 39s | trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 38s | trunk passed | |
| +1 :green_heart: | shadedclient | 22m 52s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 1m 23s | the patch passed | |
| +1 :green_heart: | compile | 1m 33s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javac | 1m 33s | the patch passed | |
| +1 :green_heart: | compile | 1m 24s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 1m 24s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 1m 2s | /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 261 unchanged - 1 fixed = 263 total (was 262) |
| +1 :green_heart: | mvnsite | 1m 27s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 53s | the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 | |
| +1 :green_heart: | javadoc | 1m 31s | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 22s | the patch passed | |
| +1 :green_heart: | shadedclient | 23m 5s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 256m 25s | hadoop-hdfs in the patch passed. | |
| +1 :green_heart: | asflicense | 0m 59s | The patch does not generate ASF License warnings. | |
| 366m 30s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/8/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4744 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux ebae8950abdc 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 823727664b94974f96c4e4bc12ffdff3462dc56f |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/8/testReport/ |
| Max. process+thread count | 2898 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/8/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 50s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 38m 59s | trunk passed | |
| +1 :green_heart: | compile | 1m 47s | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 1m 34s | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 23s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 43s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 21s | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 1m 37s | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 43s | trunk passed | |
| +1 :green_heart: | shadedclient | 23m 1s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 1m 27s | the patch passed | |
| +1 :green_heart: | compile | 1m 27s | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javac | 1m 27s | the patch passed | |
| +1 :green_heart: | compile | 1m 23s | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 1m 23s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 1m 1s | /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 285 unchanged - 1 fixed = 286 total (was 286) |
| +1 :green_heart: | mvnsite | 1m 27s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 59s | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 1m 33s | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 30s | the patch passed | |
| +1 :green_heart: | shadedclient | 22m 22s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 262m 4s | /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 19s | The patch does not generate ASF License warnings. | |
| 372m 35s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.hdfs.TestReconstructStripedFile |
| hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/10/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4744 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 4c4ce04c083f 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 58002d0e78874a22190a43904eefae665aa1d983 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/10/testReport/ |
| Max. process+thread count | 3416 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/10/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 1m 41s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 1s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 39m 0s | trunk passed | |
| +1 :green_heart: | compile | 1m 38s | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 1m 34s | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 18s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 47s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 21s | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 1m 37s | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 47s | trunk passed | |
| +1 :green_heart: | shadedclient | 23m 39s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 1m 27s | the patch passed | |
| +1 :green_heart: | compile | 1m 29s | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javac | 1m 29s | the patch passed | |
| +1 :green_heart: | compile | 1m 20s | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 1m 20s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| -0 :warning: | checkstyle | 1m 3s | /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 285 unchanged - 1 fixed = 286 total (was 286) |
| +1 :green_heart: | mvnsite | 1m 28s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 58s | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 1m 33s | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 33s | the patch passed | |
| +1 :green_heart: | shadedclient | 22m 29s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 265m 32s | /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 15s | The patch does not generate ASF License warnings. | |
| 377m 14s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.hdfs.TestReconstructStripedFile |
| hadoop.hdfs.TestDFSStripedOutputStreamWithFailure |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/9/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4744 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux b1d08172cce1 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 58002d0e78874a22190a43904eefae665aa1d983 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/9/testReport/ |
| Max. process+thread count | 3619 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/9/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 48s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 41m 58s | trunk passed | |
| +1 :green_heart: | compile | 1m 36s | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 1m 27s | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 17s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 41s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 18s | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 1m 42s | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 43s | trunk passed | |
| +1 :green_heart: | shadedclient | 26m 23s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 1m 23s | the patch passed | |
| +1 :green_heart: | compile | 1m 28s | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javac | 1m 28s | the patch passed | |
| +1 :green_heart: | compile | 1m 20s | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 1m 20s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 1m 1s | hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 284 unchanged - 1 fixed = 284 total (was 285) | |
| +1 :green_heart: | mvnsite | 1m 25s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 59s | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 1m 33s | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 31s | the patch passed | |
| +1 :green_heart: | shadedclient | 26m 31s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 344m 50s | hadoop-hdfs in the patch passed. | |
| +1 :green_heart: | asflicense | 0m 56s | The patch does not generate ASF License warnings. | |
| 464m 27s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/11/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4744 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux db61cc147d3f 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 43c73b2081d6a34ca8b14eb86eb7e342b67e2c0a |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/11/testReport/ |
| Max. process+thread count | 2242 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/11/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 41s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 1s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 1s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 38m 5s | trunk passed | |
| +1 :green_heart: | compile | 1m 36s | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 1m 35s | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 15s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 40s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 24s | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 1m 37s | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 38s | trunk passed | |
| +1 :green_heart: | shadedclient | 22m 47s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| -1 :x: | mvninstall | 1m 7s | /patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs in the patch failed. |
| -1 :x: | compile | 1m 12s | /patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt | hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | javac | 1m 12s | /patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt | hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | compile | 1m 8s | /patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt | hadoop-hdfs in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. |
| -1 :x: | javac | 1m 8s | /patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt | hadoop-hdfs in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 0m 59s | hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 284 unchanged - 1 fixed = 284 total (was 285) | |
| -1 :x: | mvnsite | 1m 10s | /patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs in the patch failed. |
| +1 :green_heart: | javadoc | 0m 55s | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 1m 32s | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| -1 :x: | spotbugs | 1m 15s | /patch-spotbugs-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs in the patch failed. |
| -1 :x: | shadedclient | 15m 51s | patch has errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 1m 20s | /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs in the patch failed. |
| +1 :green_heart: | asflicense | 0m 38s | The patch does not generate ASF License warnings. | |
| 94m 17s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/12/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4744 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 558f863b58f6 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b3fb76fd255e2274fef3e063c7ab0b14f60f9ebf |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/12/testReport/ |
| Max. process+thread count | 749 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/12/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:broken_heart: -1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 43s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 43m 37s | trunk passed | |
| +1 :green_heart: | compile | 1m 48s | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 1m 38s | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 28s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 46s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 27s | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 1m 54s | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 4m 15s | trunk passed | |
| +1 :green_heart: | shadedclient | 25m 9s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| -1 :x: | mvninstall | 0m 29s | /patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs in the patch failed. |
| -1 :x: | compile | 0m 26s | /patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt | hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | javac | 0m 26s | /patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt | hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | compile | 0m 27s | /patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt | hadoop-hdfs in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. |
| -1 :x: | javac | 0m 27s | /patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt | hadoop-hdfs in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 1m 1s | hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 284 unchanged - 1 fixed = 284 total (was 285) | |
| -1 :x: | mvnsite | 0m 26s | /patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs in the patch failed. |
| -1 :x: | javadoc | 0m 27s | /patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt | hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | javadoc | 0m 24s | /patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt | hadoop-hdfs in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. |
| -1 :x: | spotbugs | 0m 26s | /patch-spotbugs-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs in the patch failed. |
| -1 :x: | shadedclient | 12m 11s | patch has errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| -1 :x: | unit | 0m 30s | /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | hadoop-hdfs in the patch failed. |
| +1 :green_heart: | asflicense | 0m 41s | The patch does not generate ASF License warnings. | |
| 96m 32s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/13/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4744 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 2979a18966d8 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e821c5e5f48c1b2099445b769e9602cc227fd3d7 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/13/testReport/ |
| Max. process+thread count | 660 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/13/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
:confetti_ball: +1 overall
| Vote | Subsystem | Runtime | Logfile | Comment |
|---|---|---|---|---|
| +0 :ok: | reexec | 0m 40s | Docker mode activated. | |
| _ Prechecks _ | ||||
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | |
| +0 :ok: | codespell | 0m 0s | codespell was not available. | |
| +0 :ok: | detsecrets | 0m 0s | detect-secrets was not available. | |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | |
| _ trunk Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 40m 0s | trunk passed | |
| +1 :green_heart: | compile | 1m 42s | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | compile | 1m 35s | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | checkstyle | 1m 14s | trunk passed | |
| +1 :green_heart: | mvnsite | 1m 40s | trunk passed | |
| +1 :green_heart: | javadoc | 1m 17s | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 1m 42s | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 56s | trunk passed | |
| +1 :green_heart: | shadedclient | 24m 28s | branch has no errors when building and testing our client artifacts. | |
| _ Patch Compile Tests _ | ||||
| +1 :green_heart: | mvninstall | 1m 22s | the patch passed | |
| +1 :green_heart: | compile | 1m 26s | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javac | 1m 26s | the patch passed | |
| +1 :green_heart: | compile | 1m 22s | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | javac | 1m 22s | the patch passed | |
| +1 :green_heart: | blanks | 0m 0s | The patch has no blanks issues. | |
| +1 :green_heart: | checkstyle | 1m 2s | hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 284 unchanged - 1 fixed = 284 total (was 285) | |
| +1 :green_heart: | mvnsite | 1m 29s | the patch passed | |
| +1 :green_heart: | javadoc | 0m 59s | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 | |
| +1 :green_heart: | javadoc | 1m 35s | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 | |
| +1 :green_heart: | spotbugs | 3m 46s | the patch passed | |
| +1 :green_heart: | shadedclient | 24m 45s | patch has no errors when building and testing our client artifacts. | |
| _ Other Tests _ | ||||
| +1 :green_heart: | unit | 245m 21s | hadoop-hdfs in the patch passed. | |
| +1 :green_heart: | asflicense | 0m 57s | The patch does not generate ASF License warnings. | |
| 359m 34s |
| Subsystem | Report/Notes |
|---|---|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/14/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4744 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 942c597e4795 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8cd18a15b892f6b949d24edfab335255a28f5eee |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/14/testReport/ |
| Max. process+thread count | 3468 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4744/14/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
@xkrogen Sir, can you help me finally review it?
@ashutoshcipher @tomscut @ayushtkn @Hexiaoqiao Sir, can help me to double-review it when you are available?
