spring-cloud-dataflow icon indicating copy to clipboard operation
spring-cloud-dataflow copied to clipboard

In case of SCDF Local Deployment, if the SCDF server goes down in between job execution, the STATE of job gets stuck at STARTED even after restart.

Open PSHREYASHOLLA opened this issue 2 years ago • 4 comments
trafficstars

This is in reference to https://github.com/spring-cloud/spring-cloud-dataflow/issues/5498. Trying out the sample BillRun application available as part of https://dataflow.spring.io/docs/batch-developer-guides/batch/data-flow-spring-batch/.

Now we want to check the behavior of a job if the SCDF server crashes.

So for the same I have modified BillProcessor class to add a delay of 2 minutes before the 3rd record is added to the database like, public class BillProcessor implements ItemProcessor<Usage, Bill> {

@Override public Bill process(Usage usage) { if(usage.getFirstName().equals("michael")) { try { Thread.sleep(120000); } catch (InterruptedException e) { // TODO Auto-generated catch block e.printStackTrace(); } } Double billAmount = usage.getDataUsage() * .001 + usage.getMinutes() * .01; return new Bill(usage.getId(), usage.getFirstName(), usage.getLastName(), usage.getDataUsage(), usage.getMinutes(), billAmount); } }

Now we launch the billRun task and the batch inserts the first 2 records into the DB and goes to sleep for 2 minutes, image

We bring down the server now. After server restart the job execution status remains as STARTED, image

So now its neither continuing on server startup with this status nor its allowing me to restart as the button is disabled, image

So there should be a way to correct the STATUS of the job and option to restart the same from where it was left.

PSHREYASHOLLA avatar Oct 10 '23 05:10 PSHREYASHOLLA

When running locally dataflow launches the application and when stopping dataflow, this also stops the task.

corneil avatar Oct 11 '23 09:10 corneil

@corneil , But task stays in STARTED status and it does not allow to restart the task.

PSHREYASHOLLA avatar Oct 11 '23 11:10 PSHREYASHOLLA

This would be an API addition to dataflow to set the state of the task execution to the desired state.

cppwfs avatar Oct 11 '23 13:10 cppwfs

This would be an API addition to dataflow to set the state of the task execution to the desired state. >>>> Can you please tell me if it will be taken up any time soon?

PSHREYASHOLLA avatar Oct 12 '23 04:10 PSHREYASHOLLA