iotdb icon indicating copy to clipboard operation
iotdb copied to clipboard

[Bug] 0.14.0版本,集群部署iotdb,不定时出现数据库无法操作的问题,报错Sink handle is aborted;Driver was interrupted

Open qjwzero opened this issue 2 years ago • 2 comments

Search before asking

  • [X] I searched in the issues and found nothing similar.

Version

0.14.0

Describe the bug and provide the minimal reproduce step

集群部署iotdb:1号机器:ConfigNode,DataNode;2号机器:DataNode;3号机器:DataNode 问题描述:不定时出现数据库无法操作的问题 3号机器的DataNode出现error log: 2023-11-03 09:59:55,593 [Query-Worker-Thread-10$20231103_015954_06075_1.2.0] ERROR o.a.i.d.m.e.s.AbstractDriverThread:76 - [ExecuteFailed] java.lang.IllegalStateException: Sink handle is aborted. at org.apache.iotdb.db.mpp.execution.exchange.SinkHandle.checkState(SinkHandle.java:351) at org.apache.iotdb.db.mpp.execution.exchange.SinkHandle.isFull(SinkHandle.java:132) at org.apache.iotdb.db.mpp.execution.driver.Driver.processInternal(Driver.java:181) at org.apache.iotdb.db.mpp.execution.driver.Driver.lambda$processFor$1(Driver.java:125) at org.apache.iotdb.db.mpp.execution.driver.Driver.tryWithLock(Driver.java:270) at org.apache.iotdb.db.mpp.execution.driver.Driver.processFor(Driver.java:118) at org.apache.iotdb.db.mpp.execution.schedule.DriverTaskThread.execute(DriverTaskThread.java:63) at org.apache.iotdb.db.mpp.execution.schedule.AbstractDriverThread.run(AbstractDriverThread.java:70) 2023-11-03 09:59:55,882 [Query-Worker-Thread-10$20231103_015954_06262_1.2.0] ERROR o.a.i.d.m.e.s.AbstractDriverThread:76 - [ExecuteFailed] java.lang.RuntimeException: Driver was interrupted at org.apache.iotdb.db.mpp.execution.driver.Driver.processInternal(Driver.java:204) at org.apache.iotdb.db.mpp.execution.driver.Driver.lambda$processFor$1(Driver.java:125) at org.apache.iotdb.db.mpp.execution.driver.Driver.tryWithLock(Driver.java:270) at org.apache.iotdb.db.mpp.execution.driver.Driver.processFor(Driver.java:118) at org.apache.iotdb.db.mpp.execution.schedule.DriverTaskThread.execute(DriverTaskThread.java:63) at org.apache.iotdb.db.mpp.execution.schedule.AbstractDriverThread.run(AbstractDriverThread.java:70) Suppressed: java.lang.IllegalStateException: Sink handle is aborted. at org.apache.iotdb.db.mpp.execution.exchange.SinkHandle.checkState(SinkHandle.java:351) at org.apache.iotdb.db.mpp.execution.exchange.SinkHandle.isFull(SinkHandle.java:132) at org.apache.iotdb.db.mpp.execution.driver.Driver.processInternal(Driver.java:181) ... 5 common frames omitted Caused by: java.lang.Exception: Interrupted By at java.lang.Thread.getStackTrace(Thread.java:1559) at org.apache.iotdb.db.mpp.execution.driver.Driver$DriverLock.interruptCurrentOwner(Driver.java:440) at org.apache.iotdb.db.mpp.execution.driver.Driver.close(Driver.java:148) at org.apache.iotdb.db.mpp.execution.fragment.FragmentInstanceExecution.lambda$initialize$0(FragmentInstanceExecution.java:120) at org.apache.iotdb.db.mpp.execution.StateMachine.fireStateChangedListener(StateMachine.java:241) at org.apache.iotdb.db.mpp.execution.StateMachine.lambda$fireStateChanged$0(StateMachine.java:234) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

问题在重启集群的DataNode后解决,但仍然会不定时出现相同的问题

What did you expect to see?

此问题的解决方法

What did you see instead?

2023-11-03 09:59:55,593 [Query-Worker-Thread-10$20231103_015954_06075_1.2.0] ERROR o.a.i.d.m.e.s.AbstractDriverThread:76 - [ExecuteFailed] java.lang.IllegalStateException: Sink handle is aborted. at org.apache.iotdb.db.mpp.execution.exchange.SinkHandle.checkState(SinkHandle.java:351) at org.apache.iotdb.db.mpp.execution.exchange.SinkHandle.isFull(SinkHandle.java:132) at org.apache.iotdb.db.mpp.execution.driver.Driver.processInternal(Driver.java:181) at org.apache.iotdb.db.mpp.execution.driver.Driver.lambda$processFor$1(Driver.java:125) at org.apache.iotdb.db.mpp.execution.driver.Driver.tryWithLock(Driver.java:270) at org.apache.iotdb.db.mpp.execution.driver.Driver.processFor(Driver.java:118) at org.apache.iotdb.db.mpp.execution.schedule.DriverTaskThread.execute(DriverTaskThread.java:63) at org.apache.iotdb.db.mpp.execution.schedule.AbstractDriverThread.run(AbstractDriverThread.java:70) 2023-11-03 09:59:55,882 [Query-Worker-Thread-10$20231103_015954_06262_1.2.0] ERROR o.a.i.d.m.e.s.AbstractDriverThread:76 - [ExecuteFailed] java.lang.RuntimeException: Driver was interrupted at org.apache.iotdb.db.mpp.execution.driver.Driver.processInternal(Driver.java:204) at org.apache.iotdb.db.mpp.execution.driver.Driver.lambda$processFor$1(Driver.java:125) at org.apache.iotdb.db.mpp.execution.driver.Driver.tryWithLock(Driver.java:270) at org.apache.iotdb.db.mpp.execution.driver.Driver.processFor(Driver.java:118) at org.apache.iotdb.db.mpp.execution.schedule.DriverTaskThread.execute(DriverTaskThread.java:63) at org.apache.iotdb.db.mpp.execution.schedule.AbstractDriverThread.run(AbstractDriverThread.java:70) Suppressed: java.lang.IllegalStateException: Sink handle is aborted. at org.apache.iotdb.db.mpp.execution.exchange.SinkHandle.checkState(SinkHandle.java:351) at org.apache.iotdb.db.mpp.execution.exchange.SinkHandle.isFull(SinkHandle.java:132) at org.apache.iotdb.db.mpp.execution.driver.Driver.processInternal(Driver.java:181) ... 5 common frames omitted Caused by: java.lang.Exception: Interrupted By at java.lang.Thread.getStackTrace(Thread.java:1559) at org.apache.iotdb.db.mpp.execution.driver.Driver$DriverLock.interruptCurrentOwner(Driver.java:440) at org.apache.iotdb.db.mpp.execution.driver.Driver.close(Driver.java:148) at org.apache.iotdb.db.mpp.execution.fragment.FragmentInstanceExecution.lambda$initialize$0(FragmentInstanceExecution.java:120) at org.apache.iotdb.db.mpp.execution.StateMachine.fireStateChangedListener(StateMachine.java:241) at org.apache.iotdb.db.mpp.execution.StateMachine.lambda$fireStateChanged$0(StateMachine.java:234) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Anything else?

No response

Are you willing to submit a PR?

  • [X] I'm willing to submit a PR!

qjwzero avatar Nov 03 '23 08:11 qjwzero

Hi, this is your first issue in IoTDB project. Thanks for your report. Welcome to join the community!

github-actions[bot] avatar Nov 03 '23 08:11 github-actions[bot]

0.14.0 版本已经比较老了,并且0.14只是review版本,建议升级到最新的1.2.2 正式版本。大概率已经解决了。

HTHou avatar Nov 03 '23 09:11 HTHou