spring-cloud-dataflow icon indicating copy to clipboard operation
spring-cloud-dataflow copied to clipboard

Allow a task to be launched across multiple platforms

Open damthemad opened this issue 2 years ago • 3 comments
trafficstars

Description: I'm not sure if this is a bug or feature.
For me it sounds like a bug, but there is a specific exception with a specific message which shows that this is intentional

When you try to run the same task on two differents platforms (using spring.cloud.dataflow.task.platformName property), scdf throws an exception (IllegalStateException), and a 500 http error code api response, with a message that explains that this is not possible (see screenshot).

Release versions: 2.10.3

Steps to reproduce: Declare two platforms in scdf

          task:
            platform:
              kubernetes:
                accounts:
                  platform1:
                    namespace: "xxx"
                    imagePullPolicy: "xxx"
                    imagePullSecret: "xxx"

                  platform2:
                    namespace: "yyy"
                    imagePullPolicy: "yyy"
                    imagePullSecret: "yyy"

Launch one task on the first platform (via ihm). for eg test-task. on platform1 Try to launch the same task on the second platform, platform2

Screenshots: image

Additional context: i can see that scdf reads table TASK_DEPLOYMENT and blocks if there is a line with TASK_DEFINITION_NAME = "test-task" and PLATFORM_NAME = "platform1".

damthemad avatar Nov 08 '23 13:11 damthemad

https://github.com/spring-cloud/spring-cloud-dataflow/blob/main/spring-cloud-dataflow-server-core/src/main/java/org/springframework/cloud/dataflow/server/service/impl/DefaultTaskExecutionService.java

damthemad avatar Nov 08 '23 13:11 damthemad

That task definition is associated with a platform upon the first launch. This allows users to get logs for the task executions along with other services for that platform. I will change this to a feature request, such that a platform is associated with a task execution vs a task definition.

cppwfs avatar Nov 09 '23 11:11 cppwfs

Thanks a lot.

damthemad avatar Nov 09 '23 13:11 damthemad