Execution is extremely slow.Is the conductor suitable for real-time scenarios?

[lianjunwei]

Describe the bug Running the workflow takes too long. Is the conductor suitable for real-time scenarios?I hope to get guidance from friends in the community. Mine has an api that turned out to take 5~6 seconds. After I changed the workflow execution it took up to 1 minute and 25 seconds.

Details Conductor version: 3.10.3 Persistence implementation: Postgres Queue implementation: Postgres Lock: Redis Workflow definition: Task definition: Event handler definition:

To Reproduce

1. An example of a very slow factor:： a.the switch decision takes 1 to 2 seconds. b.There is a simple task. The worker actually takes 102 ms, but the conductor takes 577 ms (Screenshot below.).

2. contions: My workflow has 8 simple workers with 7 sub-workflows.

Expected behavior A clear and concise description of what you expected to happen. How to reduce the execution time of workflows and tasks. I hope friends in the community can give some suggestions.

Screensh ots

Additional context Add any other context about the problem here.

1. start WorkFlow codes:

public void startWorkFlow(ControlInstruction control) {
        String conductorServerURL = "http://10.190.X.X:8080/api/";
        WorkflowExecutor executor = new WorkflowExecutor(conductorServerURL, 2);
        executor.initWorkers(InstructionFlowService.class.getPackageName());
        ConductorWorkflow<Object> workflow = this.buildExecuteCommandFlow(executor);
        try {
            CompletableFuture<Workflow> executionFuture = workflow.execute(control);
            Workflow run = executionFuture.get(60, TimeUnit.SECONDS);

            log.info("Workflow Id: " + run.getWorkflowId());
            log.info("Workflow Status: " + run.getStatus());
            log.info("Workflow Output: " + run.getOutput());
        } catch (Exception e) {
            e.printStackTrace();
        }
        log.info("Done");
    }

2. pom.xml is configured as follows:

<dependency>
    <groupId>com.netflix.conductor</groupId>
    <artifactId>conductor-client-spring</artifactId>
    <version>3.10.3</version>
</dependency>

 <dependency>
    <groupId>com.netflix.conductor</groupId>
    <artifactId>conductor-java-sdk</artifactId>
    <version>3.10.3</version>
 </dependency>  

3.The yml file configuration of the microservice where the worker is located is as follows:

conductor:
  client:
    rootUri: http://10.190.X.X:8080/api/
    threadCount: 20
  worker:
    pollingInterval: 1

4. Other similar feedback, see #3034 and #3162 . It doesn't seem to be resolved.

[aravindanr]

Conductor in its current form is not built with real-time scenarios. However, in this case, i would check your connectivity from Conductor -> Postgres and see if there are delays there. We can't see a scenario where a SWITCH task is slower than a SIMPLE task. SWITCH tasks in our deployment is executed in the order of few milliseconds.

[lianjunwei]

Conductor in its current form is not built with real-time scenarios. However, in this case, i would check your connectivity from Conductor -> Postgres and see if there are delays there. We can't see a scenario where a SWITCH task is slower than a SIMPLE task. SWITCH tasks in our deployment is executed in the order of few milliseconds.

@aravindanr I now use version 3.11.0. The primary storage is Redis. The following is the duration of my 3 minute pressure test (qps=20). I observed that the memory and cpu utilization of the conductor cluster are not high, and the performance is good. Why does the conductor's SWITCH task execute so slowly? Can you give me some optimization suggestions?

[github-actions[bot]]

This issue is stale, because it has been open for 45 days with no activity. Remove the stale label or comment, or this will be closed in 7 days.

[github-actions[bot]]

This issue was closed, because it has been stalled for 7 days with no activity.

[garathi]

Hello @lianjunwei Are you able to resolve the issue ? We have similar issue where the task is taking almost 2-3 sec to pick.