dbt-clickhouse icon indicating copy to clipboard operation
dbt-clickhouse copied to clipboard

Python models support

Open yoavo-datricks opened this issue 1 year ago • 4 comments

Hi, I was wondering if there is a plan to add python models support. We needed to create a model with a recursive CTE and were hoping we could use python since there is no native support for this in clickhouse. Thanks!

yoavo-datricks avatar Nov 14 '22 21:11 yoavo-datricks

That isn't on the short term roadmap, and I don't have any sense of how complex it will be (since apparently there isn't any high level documentation on Python models). As always pull requests are welcome, and priorities can change based on user interest.

genzgd avatar Nov 14 '22 23:11 genzgd

+1 for this functionality, it can bring alot of power for dbt with clickhouse

oleg-savko avatar Nov 18 '22 12:11 oleg-savko

From what I can see in the updated DBT documentation, Python models run on the "platform", which in this case would be ClickHouse. At the moment there is no way to hook into any Python like runtime that interacts directly with ClickHouse. (The supported platforms, SnowFlake, DataBricks, and BigQuery already have integrated Python support -- for Snowflake it's in a separate "Public Preview" offering, for DataBricks and BigQuery it's Spark/PySpark). I don't see Python functionality on the roadmap for core ClickHouse any time soon, so I'm closing this. If something like PySpark gets traction in ClickHouse we will revisit the issue.

genzgd avatar Dec 16 '22 03:12 genzgd

Reopening this so we don't get more duplicates.

genzgd avatar Dec 28 '23 13:12 genzgd