conductor
conductor copied to clipboard
Netflix
SWITCH task should COMPLETE after decision cases are done
What bothers me SWITCH task is marked as COMPLETEd immediately after processing expression. With this behaviour it is dangerous to use it in DO_WHILE task, where every iteration is marked as completed (and next one is started) when every task from loopTasks are completed. Similar problem shows for FORK_JOIN where JOIN task has SWITCH in joinOn section but does not wait for it's decision cases.
Example As an example you may run following workflow
{
"name": "ADHoc-loop-test",
"version": 1,
"input": {
"list": [
{
"number": 1
},
{
"number": 2
},
{
"number": 3
}
]
},
"workflowDef": {
"name": "ADHoc-loop-test",
"description": "Program for testing loop behaviour",
"version": 1,
"schemaVersion": 2,
"ownerEmail": "[email protected]",
"tasks": [
{
"name": "LoopTask",
"taskReferenceName": "LoopTask",
"type": "DO_WHILE",
"inputParameters": {
"list": "${workflow.input.list}"
},
"loopCondition": "$.LoopTask['iteration'] < $.list.length",
"loopOver": [
{
"name": "GetNumberAtIndex",
"taskReferenceName": "GetNumberAtIndex",
"type": "INLINE",
"inputParameters": {
"evaluatorType": "javascript",
"list": "${workflow.input.list}",
"iterator": "${LoopTask.output.iteration}",
"expression": "function getElement() { return $.list.get($.iterator - 1); } getElement();"
}
},
{
"name": "SwitchTask",
"taskReferenceName": "SwitchTask",
"type": "SWITCH",
"evaluatorType": "javascript",
"inputParameters": {
"param": "${GetNumberAtIndex.output.result.number}"
},
"expression": "$.param > 0",
"decisionCases": {
"true": [
{
"name": "WaitTask",
"taskReferenceName": "WaitTask",
"type": "WAIT",
"inputParameters": {
"duration": "10 secs"
}
},
{
"name": "ComputeNumber",
"taskReferenceName": "ComputeNumber",
"type": "INLINE",
"inputParameters": {
"evaluatorType": "javascript",
"number": "${GetNumberAtIndex.output.result.number}",
"expression": "function compute() { return $.number+10; } compute();"
}
}
]
}
}
]
}
]
}
}
From above execution I expect to process every number sequentially but the third one is processed three times instead.
Expected behavior It would be great if SWITCH task completed after all of its decisionCases and defaultCase are finished.
Hello @sebimal I tested the above workflow, and the issue I saw is that the switch task is evaluated to true, but the waitTask and computeNumber tasks are not scheduled correctly. I suspect it has something to do with how do_while spawn up next task. Please feel free to submit a PR if you have time to look into fixing the issue.
Hi @sebimal , Thanks for reporting the issue. Looks like some issue in do_while logic. @jxu-nflx , I will raise PR in some time to fix this.
This is same happening to me. Is there any fix associated to this or workaround
This is same happening to me. Is there any fix associated to this or workaround
I used SUB_WORKFLOW to wrap SWITCH with all of its branched behaviour.
Workarround for my previous example:
{
"name": "ADHoc-loop-test",
"version": 1,
"input": {
"list": [
{
"number": 1
},
{
"number": 2
},
{
"number": 3
}
]
},
"workflowDef": {
"name": "ADHoc-loop-test",
"description": "Program for testing loop behaviour",
"version": 1,
"schemaVersion": 2,
"ownerEmail": "[email protected]",
"tasks": [
{
"name": "LoopTask",
"taskReferenceName": "LoopTask",
"type": "DO_WHILE",
"inputParameters": {
"list": "${workflow.input.list}"
},
"loopCondition": "$.LoopTask['iteration'] < $.list.length",
"loopOver": [
{
"name": "GetNumberAtIndex",
"taskReferenceName": "GetNumberAtIndex",
"type": "INLINE",
"inputParameters": {
"evaluatorType": "javascript",
"list": "${workflow.input.list}",
"iterator": "${LoopTask.output.iteration}",
"expression": "function getElement() { return $.list.get($.iterator - 1); } getElement();"
}
},
{
"name": "SwitchSubWorkflow",
"taskReferenceName": "SwitchSubWorkflow",
"type": "SUB_WORKFLOW",
"inputParameters": {
"param": "${GetNumberAtIndex.output.result.number}"
},
"subWorkflowParam": {
"name": "SwitchWrapper",
"version": 1,
"workflowDefinition": {
"name": "SwitchWrapper",
"version": 1,
"ownerEmail": "[email protected]",
"tasks": [
{
"name": "SwitchTask",
"taskReferenceName": "SwitchTask",
"type": "SWITCH",
"evaluatorType": "javascript",
"inputParameters": {
"param": "${workflow.input.param}"
},
"expression": "$.param > 0",
"decisionCases": {
"true": [
{
"name": "WaitTask",
"taskReferenceName": "WaitTask",
"type": "WAIT",
"inputParameters": {
"duration": "10 secs"
}
},
{
"name": "ComputeNumber",
"taskReferenceName": "ComputeNumber",
"type": "INLINE",
"inputParameters": {
"evaluatorType": "javascript",
"number": "${workflow.input.param}",
"expression": "function compute() { return $.number+10; } compute();"
}
}
]
}
}
]
}
}
}
]
}
]
}
}