Delta Live Tables

[dwolfeu]

Describe the feature

Create and orchestrate Delta Live Tables using dbt-databricks

Describe alternatives you've considered

Delta Live Tables can be set up using the Databricks UI, but this misses all the excellent features of dbt.

Additional context

I do not know whether this idea is feasible, since Delta Live Tables function only in tandem with Databricks pipelines, which goes beyond CREATE TABLE.

Who will this benefit?

Anyone who likes both dbt and Delta Live Tables, in particular teams with existing dbt-databricks projects who want to migrate to Delta Live Tables.

Are you interested in contributing this feature?

Potentially, but I am a complete newbie to this repo.

[dcaribou]

dbt + DLT sounds like very powerful combination. I don't know how feasible this is, but I ❤️ the idea.

[JCZuurmond]

Is there an update on this enhancement? I am curious!

[bilalaslamseattle]

@JCZuurmond please tell me more ... what all would you like to be able to do?

[Angelo-Baruffi]

@bilalaslamseattle the ideia is to use all the power of dbt framework, like tests, models, code structure, jinja, dbt utils, etc.. and combine with the power of DLT using streaming data easily. As @dwolfeu said, its a good integration for teams that already work with dbt

[bilalaslamseattle]

@Angelo-Baruffi makes a lot of sense. Thank you!

[jc-ludia]

Plus one to what Angelo mentionned so far!

[followingell]

Would also love to see this.

[yang6n]

DLT becomes the mainstream of Spark nowadays, and adding support to DLT in dbt is one of the most want features from our organization.

[benc-db]

We added initial support in 1.6.0, but significant improvements are coming during the 1.7.x timeframe.