[Bug] The Bug of Parameter passing feature is unavailable in SQL Task still exists in 3.3.2

[yaowanli]

Search before asking

• [x] I had searched in the issues and found no similar issues.

What happened

Parameter passing feature is unavailable.

ALL task and stream

task1-output the value 'order_no' as Parameter

task2-use the Parameter 'order_no',produce the Parameter 'output'

task3-use the Parameter 'output'

task1-result

task2-result

task3-result

What you expected to happen

After upstream tasks complete execution, parameters are not merged into global parameters, causing the parameter passing functionality to be unavailable. This feature needs to be restored.

How to reproduce

Parameter passing missing

Anything else

No response

Version

3.3.2

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [x] I agree to follow this project's Code of Conduct

[Mrhs121]

You can try adding a variable of type in in your task3, and try it again.

I just ran the following workflow and saw the Task 2 SQL task pick up the upstream shell task output and return the expected results.

[yaowanli]

Oh, i get you!in this way ,i can get the variable from the upstream node,but i have to explicitly add IN type variables every time,it seems a bit trouble. Why can't I declare the OUT type variable first and then add it to the local variables so that downstream nodes can use it directly?That's how it's demonstrated in the official documentation. Or could it be optimized by allowing selection of an existing local variable from a dropdown menu?

[Mrhs121]

Oh, i get you!in this way ,i can get the variable from the upstream node,but i have to explicitly add IN type variables every time,it seems a bit trouble. Why can't I declare the OUT type variable first and then add it to the local variables so that downstream nodes can use it directly?That's how it's demonstrated in the official documentation. Or could it be optimized by allowing selection of an existing local variable from a dropdown menu?

I found that the behavior of Parameter passing has changed in version 3.3.x. In old version 3.2.x, as shown in the above example, task2 can obtain the upstream output without configuring a variable of type in. On the contrary, when a variable of type in is set, the upstream output cannot be obtained.

In version 3.3.x, the following processing logic has been added. For example, in the above example, only when local var task_type is configured in task2 will the upstream varPools value(internal_table_creation_task) be used to override the value of local var task_type

https://github.com/apache/dolphinscheduler/blob/797014807f774ff17f617db13b9e6acd64164f6b/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/expand/CuringParamsServiceImpl.java#L226-L235

[yaowanli]

Thanks for your reply. If the parameter passing logic has indeed been modified, then I have no objection. I suggest updating the description and images regarding "parameter passing" in the official documentation: When downstream nodes use local variables, the variables need to be explicitly configured.

[SbloodyS]

Thanks for your reply. If the parameter passing logic has indeed been modified, then I have no objection. I suggest updating the description and images regarding "parameter passing" in the official documentation: When downstream nodes use local variables, the variables need to be explicitly configured.

You are right. Would you like to update the docs?

[yaowanli]

Oh, it's my pleasure. I'll try to revise the document in the next few days.

[SbloodyS]

Oh, it's my pleasure. I'll try to revise the document in the next few days.

I've assigned to you. Looking forward to your contribution.

[abangv]

Oh, i get you!in this way ,i can get the variable from the upstream node,but i have to explicitly add IN type variables every time,it seems a bit trouble. Why can't I declare the OUT type variable first and then add it to the local variables so that downstream nodes can use it directly?That's how it's demonstrated in the official documentation. Or could it be optimized by allowing selection of an existing local variable from a dropdown menu?

I found that the behavior of Parameter passing has changed in version 3.3.x. In old version 3.2.x, as shown in the above example, task2 can obtain the upstream output without configuring a variable of type in. On the contrary, when a variable of type in is set, the upstream output cannot be obtained.

In version 3.3.x, the following processing logic has been added. For example, in the above example, only when local var task_type is configured in task2 will the upstream varPools value(internal_table_creation_task) be used to override the value of local var task_type

dolphinscheduler/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/expand/CuringParamsServiceImpl.java

Lines 226 to 235 in 7970148

if (CollectionUtils.isNotEmpty(varPools)) { // overwrite the in parameter by varPool for (Property varPool : varPools) { Property property = prepareParamsMap.get(varPool.getProp()); if (property == null || property.getDirect() != Direct.IN) { continue; } property.setValue(varPool.getValue()); } } 3.3.1需要添加以下修复的代码