dbt-sqlserver
dbt-sqlserver copied to clipboard
support Ephemeral models
from #137, here's the proposed solution
Ephemeral models
The general approach outlined in https://github.com/dbt-msft/dbt-synapse/commit/9180da5a6d092c6b4b22d5388aa6592d2b9d77c4 is still the same, though I was doing it on easy mode, and didn't actually try to solve for ephemeral model compilation :)
The mechanism is: reimplement the compiler, define a new one (like in
dbt/adapters/sqlserver/compilation.py), and reimplement methods like_inject_ctes_into_sqland_recursively_prepend_ctes.Taking a step back: In order to support ephemeral models with CTEs on SQLServer and Synapse, given that T-SQL doesn't support CTEs nested inside either CTEs or subqueries, would we have to take the same approach as the one outlined above?
Namely, instead of recursively prepending ephemeral models as CTEs to the start of the query:
with __dbt__cte__ephemeral_model as ( with my_cte as ( select 1 as id ) select * from my_cte ),ephemeral_model as ( select * from __dbt__cte__ephemeral_model ), another_cte as ( select 2 as id ) select * from ephemeral_model union all select * from another_cteRecursively prepend them as temp tables, to be executed in-transaction with the body of the query:
create table #ephemeral_model as ( with my_cte as ( select 1 as id ) select * from my_cte ); with ephemeral_model as ( select * from #ephemeral_model ), another_cte as ( select 2 as id ) select * from ephemeral_model union all select * from another_cte
I don't think we should do this :)
Newer MSFT databases should support nested CTEs (https://github.com/dbt-msft/dbt-sqlserver/issues/457). We shouldn't expose the full Compiler interface to adapters (https://github.com/dbt-labs/dbt-core/pull/9134).