spring-cloud-dataflow
spring-cloud-dataflow copied to clipboard
Running a composed task as a schedule causes an NPE
Description: We are configuring a composed task with the following settings
- Application 1: query-executor [type: Task, version: 3.x]
- Application 2: data-loader [type: Task, version: 3.x]
- Composed Task: xxx-statistics [query-executor && data-loader]
If we create the above composed task as a schedule and look at the log of the completed execution, we will see the following NullPointerException.
2024-06-18 00:30:07.944 INFO 1 --- [ main] .c.d.c.ComposedTaskStepExecutionListener : ComposedTaskStepExecutionListener supporting [query-executor, data-loader]
...
2024-06-18 00:30:15.377 INFO 1 --- [ main] o.s.c.d.c.ComposedTaskRunnerTaskListener : onTaskStartup:executionId=1534
2024-06-18 00:30:15.418 INFO 1 --- [ main] o.s.b.web.embedded.netty.NettyWebServer : Netty started on port 8080
2024-06-18 00:30:15.483 INFO 1 --- [ main] o.s.c.d.c.ComposedTaskRunner : Started ComposedTaskRunner in 13.685 seconds (JVM running for 14.578)
2024-06-18 00:30:15.490 INFO 1 --- [ main] o.s.b.a.b.JobLauncherApplicationRunner : Running default command line with: []
2024-06-18 00:30:15.491 INFO 1 --- [ main] o.s.b.c.r.s.JobRepositoryFactoryBean : No database type set, using meta data indicating: POSTGRES
2024-06-18 00:30:15.726 INFO 1 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=xxx-statistics]] launched with the following parameters: [{run.id=2}]
2024-06-18 00:30:15.746 INFO 1 --- [ main] o.s.c.t.b.l.TaskBatchExecutionListener : The job execution id 1297 was run within the task execution 1534
2024-06-18 00:30:15.799 INFO 1 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [xxx-query-executor_0]
2024-06-18 00:30:15.809 INFO 1 --- [ main] .c.d.c.ComposedTaskStepExecutionListener : beforeStep:xxx-query-executor_0:1297>>>>
2024-06-18 00:30:15.809 INFO 1 --- [ main] .c.d.c.ComposedTaskStepExecutionListener : beforeStep:xxx-query-executor_0:1297<<<
2024-06-18 00:31:30.964 INFO 1 --- [ main] .c.d.c.ComposedTaskStepExecutionListener : AfterStep processing for stepExecution xxx-query-executor_0:1297
2024-06-18 00:31:30.964 INFO 1 --- [ main] .c.d.c.ComposedTaskStepExecutionListener : AfterStep for xxx-query-executor_0:1297:xxx-query-executor:630:boot3
2024-06-18 00:31:30.965 WARN 1 --- [ main] o.s.c.d.c.TaskExplorerContainer : Cannot find TaskExplorer for boot3. Using default
2024-06-18 00:31:30.972 ERROR 1 --- [ main] o.s.batch.core.step.AbstractStep : Exception in afterStep callback in step xxx-query-executor_0 in job xxx-statistics
java.lang.NullPointerException: null
at org.springframework.cloud.dataflow.composedtaskrunner.ComposedTaskStepExecutionListener.afterStep(ComposedTaskStepExecutionListener.java:83) ~[classes/:na]
at org.springframework.batch.core.listener.CompositeStepExecutionListener.afterStep(CompositeStepExecutionListener.java:64) ~[spring-batch-core-4.3.9.jar:4.3.9]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:247) ~[spring-batch-core-4.3.9.jar:4.3.9]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:152) ~[spring-batch-core-4.3.9.jar:4.3.9]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[spring-batch-core-4.3.9.jar:4.3.9]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[spring-batch-core-4.3.9.jar:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:167) ~[spring-batch-core-4.3.9.jar:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:142) ~[spring-batch-core-4.3.9.jar:4.3.9]
at org.springframework.batch.core.job.flow.support.state.FlowState.handle(FlowState.java:56) ~[spring-batch-core-4.3.9.jar:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:167) ~[spring-batch-core-4.3.9.jar:4.3.9]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:142) ~[spring-batch-core-4.3.9.jar:4.3.9]
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:139) ~[spring-batch-core-4.3.9.jar:4.3.9]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:320) ~[spring-batch-core-4.3.9.jar:4.3.9]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:149) ~[spring-batch-core-4.3.9.jar:4.3.9]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) ~[spring-core-5.3.29.jar:5.3.29]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140) ~[spring-batch-core-4.3.9.jar:4.3.9]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
...
The good news is that the subtasks within that composed task complete without a problem. Note that this error does not occur when the composed task is run via LAUNCH TASK. There is no difference in the settings between LAUNCH TASK and SCHEDULE TASK.
Release versions:
- K8S: v1.28.9
- SCDF: 2.11.2
- CTR: 2.11.0
@wqld Please try CTR 2.11.2 or even better upgrade both SCDF and CTR to 2.11.3
@corneil The same error occurs even with CTR 2.11.2. I will consider upgrading the SCDF version later.
Can you provide the task definition you are using?
Issue closed due to inactivity. If closed in error, please let us know.