seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Bug] [zeta] When restore the job, because the reader has not been registered, the split is assigned first and the task is abnormal

Open liugddx opened this issue 2 years ago • 2 comments

close #4764

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

When restore the job, because the reader has not been registered, the split is assigned first and the task is abnormal

SeaTunnel Version

dev

SeaTunnel Config

-

Running Command

-

Error Exception

java.lang.NullPointerException
	at org.apache.seatunnel.connectors.seatunnel.kafka.source.KafkaSourceSplitEnumerator.listOffsets(KafkaSourceSplitEnumerator.java:327)
	at org.apache.seatunnel.connectors.seatunnel.kafka.source.KafkaSourceSplitEnumerator.convertToNextSplit(KafkaSourceSplitEnumerator.java:191)
	at org.apache.seatunnel.connectors.seatunnel.kafka.source.KafkaSourceSplitEnumerator.addSplitsBack(KafkaSourceSplitEnumerator.java:182)
	at org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask.addSplitsBack(SourceSplitEnumeratorTask.java:186)
	at org.apache.seatunnel.engine.server.task.operation.source.RestoredSplitOperation.lambda$run$1(RestoredSplitOperation.java:98)
	at org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:48)
	at org.apache.seatunnel.engine.server.task.operation.source.RestoredSplitOperation.run(RestoredSplitOperation.java:82)
	at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189)
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273)
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248)
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:213)
	at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.run(OperationExecutorImpl.java:411)
	at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.runOrExecute(OperationExecutorImpl.java:438)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvokeLocal(Invocation.java:601)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvoke(Invocation.java:580)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke0(Invocation.java:541)
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke(Invocation.java:241)
	at com.hazelcast.spi.impl.operationservice.impl.InvocationBuilderImpl.invoke(InvocationBuilderImpl.java:61)
	at org.apache.seatunnel.engine.server.utils.NodeEngineUtil.sendOperationToMemberNode(NodeEngineUtil.java:51)
	at org.apache.seatunnel.engine.server.execution.TaskExecutionContext.sendToMember(TaskExecutionContext.java:47)
	at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.restoreState(SourceFlowLifeCycle.java:253)
	at org.apache.seatunnel.engine.server.task.SeaTunnelTask.lambda$restoreState$14(SeaTunnelTask.java:383)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)

Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

  • [x] Yes I am willing to submit a PR!

Code of Conduct

liugddx avatar May 17 '23 15:05 liugddx

image

Cannot call assignSplit in the addSplitsBack method.

liugddx avatar May 17 '23 15:05 liugddx

cannot call assignSplit() in the open() method

liugddx avatar May 17 '23 15:05 liugddx

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] avatar Jun 21 '23 00:06 github-actions[bot]