spring-cloud-dataflow
spring-cloud-dataflow copied to clipboard
spring-cloud
JobInstanceAlreadyCompleteException due to no ordered execution-ids in OracleDB
Description:
SCDF schema created following the one present at https://github.com/spring-cloud/spring-cloud-dataflow/blob/main/spring-cloud-dataflow-server-core/src/main/java/org/springframework/cloud/dataflow/server/db/migration/oracle/V1__Initial_Setup.java
define a sequence TASK_SEQ created by the snippet below present at V1__Initial_Setup.java that didn´t has order defined
private final static String CREATE_TASK_SEQ_SEQUENCE =
"CREATE SEQUENCE TASK_SEQ START WITH 0 MINVALUE 0 MAXVALUE 9223372036854775807 NOCACHE NOCYCLE";
these sequence is created with no order that is the default value in Oracle, because of that tasks with multiple executions are no following sequence order over time in some situations and it is materialized in JobInstanceAlreadyCompleteException during the job instance.
Attached below you will find the image where you can see executions with one hour of diference being instanciated with a sequence number less than the last one (execution-id 31225 < 31245)
The problem was solved indicating the ORDER modificator in the sequence
Release versions: SCDF 2.3.1.RELEASE but it is present in actual main branch
Custom apps: N/A.
Steps to reproduce: Create a task. Execute as many executions as you want using the RunIdJobIncrementer at the batch job implementation
Screenshots:
Additional context: I think the bug is present in both spring-cloud-server and spring-cloud-task projects https://github.com/spring-cloud/spring-cloud-task/blob/main/spring-cloud-task-core/src/main/resources/org/springframework/cloud/task/schema-oracle.sql#L29, but I prefer to solve that on spring-cloud-dataflow prior to open it on spring-cloud-task.
On the other hand, what is the reason for disabling CACHE in TASK_SEQ sequence? it seems to go against OracleDB recomendations Oracle Sequences Oracle recommends using the CACHE setting to enhance performance if you are using sequences in a Real Application Clusters environment.
Also created issue https://github.com/spring-cloud/spring-cloud-task/issues/853 for task.
Sounds like the right approach. We will update the schema in scdf in the 2.10 release time frame. This is not dependent upon the upcoming releases of spring batch/task - scdf 'owns' these tables.
