dolphinscheduler icon indicating copy to clipboard operation
dolphinscheduler copied to clipboard

SQL component, parameter passing problem when the custom parameter is LIST

Open lixianli1987 opened this issue 1 year ago • 5 comments

Search before asking

  • [X] I had searched in the issues and found no similar feature requirement.

Description

1702988843880 下一个节点的配置如下 image 实际日志如下 image 请问如何可以获取到dataList的数据呢

Documentation Links

No response

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

lixianli1987 avatar Dec 19 '23 12:12 lixianli1987

Search before asking

  • [X] I had searched in the issues and found no similar feature requirement.

Description

1702988843880 The configuration of the next node is as follows image The actual log is as follows image How can I get the data of dataList?

Documentation Links

No response

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

github-actions[bot] avatar Dec 19 '23 12:12 github-actions[bot]

Right now, we only support transport the value by column. If you have two column in your table t_screen, you need to add two param.

It is a usefully feature if we can directly transport the whole output result of the task.

ruanwenjun avatar Dec 22 '23 15:12 ruanwenjun

I think this may be a large feature? For online SQL query, such as MySQL, PostgreSQL, which can return results within seconds level, we can directly use network for transport. For offline SQL query, which is calculated by distributed ETL engine, like HIVE, SPARK, FLINK, the results can be read from other source? like DFS, MQ, etc.

关于产品,目前HTTP组件不支持从上游系统传输数据过来;刚好目前我们有这个方面的需求。不知道后续的产品规划中有没得这个功能,如果没有的话,我这边能否完善这个功能点,提交到官方?

lixianli1987 avatar Dec 25 '23 02:12 lixianli1987

I've just recently started learning about this software. Can someone please help me understand how to add multiple records to a database after querying them using the SQL module? 我最近才刚开始了解这个软件,那位大佬可以帮忙告诉我 使用SQL 模块 查询除了多条数据之后 应该怎么添加到数据库

15620627645 avatar Jan 10 '24 14:01 15620627645

sql是否适合大数据量的传输,看代码数据是放到varpool这个字段中的 image,最终还会保存到库里。

yeyahui325 avatar Apr 11 '24 01:04 yeyahui325