Cannot load streaming tables

[jomach]

trafficstars

Describe the bug

This Delta Live Tables query is syntactically valid, but you must create a pipeline in order to define and populate your table.

i'm struggling integrate Delta Live Tables and dbt with one another.

Basically i'm trying to complete this tutorial, but the dbt job doesn't create tables https://www.databricks.com/blog/delivering-cost-effective-data-real-time-dbt-and-databricks

Description

I added the linked github repo to my databricks workspace.

Successfully ran the helper notebook and created and basic quering display data.

I also created a job which runs a dbt task based on the dbt project contained in the GitHub. The job exeutes dbt deps, dbt seed and dbt run.

This task completes with the message in the given picture.

As you can see the streaming table creations appear to be successful (e.g. 'OK created sql streaming_table model default.airline_trips_bronze' ) but can not be found in the later processing. Niether can i find them in the catalog explorer.

So in order to debug this issue i checked the "Query History" which tells me that the Create query ran successful with 1 returned row. As i do not know how else to access that output i simple pasted the query into the "SQL Editor" and executed it from there which returned one row saying:

"This Delta Live Tables query is syntactically valid, but you must create a pipeline in order to define and populate your table."

As shown in this blog and this demo it at least used to be possible to run these sql statements without a pipeline.

So my questions are

1. am i missing a crucial part of the tutorial?

2. do any of you guys face the same issue? (and how did you solve it)

3. are there requirements not mentioned in the tutorial?

Some basic information:

• Creating a delta live table with a Notebook and a Pipeline works just as expected

• Unity Catalog is enabled for the workspace

• Compute Cluster: Single node: Standard_DS3_v2 · 15.0 ML (includes Apache Spark 3.5.0, Scala 2.12)

• SQl Warehouse: Pro, 2X-Small like 1

[jomach]

Here is the error message from SQL:

I assume that data bricks returns a row with the message but it should return a Error

[thijs-nijhuis]

@jomach , are you sure MV's are enabled on your workspace? Not 100% sure but I think that is a requirement: https://docs.databricks.com/en/sql/user/materialized-views.html#requirements . The errors in the first screenshot seem to suggest it is not enabled. To test it, can you manually create an MV in the 'default' schema of your UC catalog? And, once created, also refresh it? The second parts is important as for me, a while ago, I had this issue where I could created them (using a PRO warehouse) but not refresh them (which requires serverless). This had to do with network access to the storage account.

[jomach]

With MV you mean Materialized View ?

[thijs-nijhuis]

With MV you mean Materialized View ?

Yes.