dolphinscheduler
dolphinscheduler copied to clipboard
Published
20 hours ago •
apache
apache
[Bug] [Task] A single task instance runs or runs backwards only once
Search before asking
- [x] I had searched in the issues and found no similar issues.
What happened
A single task instance runs or runs backwards only once
[WI-0][TI-0] - [ERROR] 2025-02-06 15:57:45.452 +0800 o.a.d.s.m.r.MasterSchedulerBootstrap:[153] - Master handle command 1160 error
org.apache.dolphinscheduler.server.master.exception.WorkflowCreateException: Create WorkflowExecuteRunnable failed
at org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteRunnableFactory.createWorkflowExecuteRunnable(WorkflowExecuteRunnableFactory.java:98)
at org.apache.dolphinscheduler.server.master.runner.MasterSchedulerBootstrap.lambda$run$0(MasterSchedulerBootstrap.java:136)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583)
at org.apache.dolphinscheduler.server.master.runner.MasterSchedulerBootstrap.run(MasterSchedulerBootstrap.java:133)
Caused by: java.lang.NullPointerException: null
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:889)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at org.apache.dolphinscheduler.service.process.ProcessServiceImpl.joinVarPool(ProcessServiceImpl.java:1038)
at org.apache.dolphinscheduler.service.process.ProcessServiceImpl.setSubProcessParam(ProcessServiceImpl.java:982)
at org.apache.dolphinscheduler.service.process.ProcessServiceImpl.handleCommand(ProcessServiceImpl.java:316)
at org.apache.dolphinscheduler.service.process.ProcessServiceImpl$$FastClassBySpringCGLIB$$9d3e18f9.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
at org.apache.dolphinscheduler.service.process.ProcessServiceImpl$$EnhancerBySpringCGLIB$$6edc3705.handleCommand(<generated>)
at org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteContextFactory.createWorkflowInstance(WorkflowExecuteContextFactory.java:81)
at org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteContextFactory.createWorkflowExecuteRunnableContext(WorkflowExecuteContextFactory.java:56)
at org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteRunnableFactory.createWorkflowExecuteRunnable(WorkflowExecuteRunnableFactory.java:83)
... 15 common frames omitted
What you expected to happen
https://github.com/apache/dolphinscheduler/blob/8ee334fc2f4803d7a628458018d82ca695b920d1/dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/utils/VarPoolUtils.java#L97
An empty array should be returned here, not null. However, the unit test specifies null. I don't know what it means.
@Test
void subtractVarPool() {
Truth.assertThat(VarPoolUtils.subtractVarPool(null, null)).isNull();
List<Property> varpool1 = Lists.newArrayList(new Property("name", Direct.OUT, DataType.VARCHAR, "tom"),
new Property("age", Direct.OUT, DataType.INTEGER, "10"));
List<Property> varpool2 = Lists.newArrayList(new Property("name", Direct.OUT, DataType.VARCHAR, "tom"));
List<Property> varpool3 = Lists.newArrayList(new Property("location", Direct.OUT, DataType.VARCHAR, "china"));
Truth.assertThat(VarPoolUtils.subtractVarPool(varpool1, Lists.newArrayList(varpool2, varpool3)))
.containsExactly(new Property("age", Direct.OUT, DataType.INTEGER, "10"));
}
How to reproduce
Process logic: A workflow is nested in another workflow through the sub_process method
- Main workflow logic
- Sub-workflow logic
Problem: When the run fails.
Enter the workflow instance
Execute backwards, or run can only be executed successfully once. No response will occur if executed again.
Anything else
No response
Version
3.2.x
Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
Code of Conduct
- [x] I agree to follow this project's Code of Conduct
Search before asking
- [x] I had searched in the issues and found no similar issues.
What happened
A single task instance runs or runs backwards only once
[WI-0][TI-0] - [ERROR] 2025-02-06 15:57:45.452 +0800 o.a.d.s.m.r.MasterSchedulerBootstrap:[153] - Master handle command 1160 error
org.apache.dolphinscheduler.server.master.exception.WorkflowCreateException: Create WorkflowExecuteRunnable failed
at org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteRunnableFactory.createWorkflowExecuteRunnable(WorkflowExecuteRunnableFactory.java:98)
at org.apache.dolphinscheduler.server.master.runner.MasterSchedulerBootstrap.lambda$run$0(MasterSchedulerBootstrap.java:136)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583)
at org.apache.dolphinscheduler.server.master.runner.MasterSchedulerBootstrap.run(MasterSchedulerBootstrap.java:133)
Caused by: java.lang.NullPointerException: null
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:889)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at org.apache.dolphinscheduler.service.process.ProcessServiceImpl.joinVarPool(ProcessServiceImpl.java:1038)
at org.apache.dolphinscheduler.service.process.ProcessServiceImpl.setSubProcessParam(ProcessServiceImpl.java:982)
at org.apache.dolphinscheduler.service.process.ProcessServiceImpl.handleCommand(ProcessServiceImpl.java:316)
at org.apache.dolphinscheduler.service.process.ProcessServiceImpl$$FastClassBySpringCGLIB$$9d3e18f9.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
at org.apache.dolphinscheduler.service.process.ProcessServiceImpl$$EnhancerBySpringCGLIB$$6edc3705.handleCommand(<generated>)
at org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteContextFactory.createWorkflowInstance(WorkflowExecuteContextFactory.java:81)
at org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteContextFactory.createWorkflowExecuteRunnableContext(WorkflowExecuteContextFactory.java:56)
at org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteRunnableFactory.createWorkflowExecuteRunnable(WorkflowExecuteRunnableFactory.java:83)
... 15 common frames omitted
What you expected to happen
@Test
void subtractVarPool() {
Truth.assertThat(VarPoolUtils.subtractVarPool(null, null)).isNull();
List<Property> varpool1 = Lists.newArrayList(new Property("name", Direct.OUT, DataType.VARCHAR, "tom"),
new Property("age", Direct.OUT, DataType.INTEGER, "10"));
List<Property> varpool2 = Lists.newArrayList(new Property("name", Direct.OUT, DataType.VARCHAR, "tom"));
List<Property> varpool3 = Lists.newArrayList(new Property("location", Direct.OUT, DataType.VARCHAR, "china"));
Truth.assertThat(VarPoolUtils.subtractVarPool(varpool1, Lists.newArrayList(varpool2, varpool3)))
.containsExactly(new Property("age", Direct.OUT, DataType.INTEGER, "10"));
}
How to reproduce
流程逻辑：一个工作流里面通过sub_process 方式，嵌套另一个工作流 主工作流逻辑
子工作流逻辑
问题：当运行过程中失败时。
进入工作流实例
执行向后运行，或运行 只能成功执行一次。 再执行就没有反应。
Anything else
No response
Version
3.2.x
Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
Code of Conduct
- [x] I agree to follow this project's Code of Conduct
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
In the dev branch, this has been refactored. https://github.com/apache/dolphinscheduler/pull/16327
https://github.com/apache/dolphinscheduler/blob/8ee334fc2f4803d7a628458018d82ca695b920d1/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessServiceImpl.java#L1033
Unable to parse here "null"
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.