dbt-fal
dbt-fal copied to clipboard
Unable to override the database when using the dbt-trino connector
Describe the bug
I'm not able to override the database
when using fal-dbt
with the dbt-trino
connector.
Your environment
- OS: Arch Linux
- Paste the following commands output:
grep dbt-fal pyproject.toml # "dbt-fal==1.4.0",
dbt --version # 1.4.0
- Adapter being used:
pdm run dbt --version 2>&1 | grep trino # - trino: 1.4.0
How to reproduce
In my dbt_project.yml
, I have this in models.metabase_reports
(note that the default database is not lakehouse
but metabase
):
experimental:
+database: lakehouse
+schema: lakehouse_gold
In the models/experimental/experimental__experimental.py
file, I have:
def model(dbt, session):
users_df = dbt.ref("all_users")
first_name_df = users_df["first_name"].drop_duplicates()
return first_name_df
When I run dbt run --select=experimental__expertimental
.
Expected behavior
A table should be created in the lakehouse.lakehouse_gold
schema within Trino.
Actual behavior
14:38 $ ./run-locally.sh -p metabase dbt run --select=experimental__experimental
13:38:24 Running with dbt=1.4.0
13:38:24 Unable to do partial parsing because of a version mismatch
13:38:25 [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
There are 1 unused configuration paths:
- models.elementary
13:38:25 Found 53 models, 30 tests, 0 snapshots, 0 analyses, 661 macros, 0 operations, 0 seed files, 26 sources, 0 exposures, 0 metrics
13:38:25
13:38:28 Concurrency: 1 threads (target='metabase_with_fal')
13:38:28
13:38:28 1 of 1 START python table model lakehouse.lakehouse_gold.experimental__experimental [RUN]
13:38:38 Unhandled error while executing target/run/metabase_reports/models/experimental/experimental__experimental.py
(trino.exceptions.TrinoUserError) TrinoUserError(type=USER_ERROR, name=NOT_FOUND, message="Schema lakehouse_gold not found", query_id=20230306_133838_00374_eem4m)
[SQL:
CREATE TABLE lakehouse_gold.__dbt_fal_temp_experimental__experimental (
first_name VARCHAR
)
]
(Background on this error at: https://sqlalche.me/e/14/f405)
13:38:38 1 of 1 ERROR creating python table model lakehouse.lakehouse_gold.experimental__experimental [ERROR in 9.78s]
13:38:38
13:38:38 Finished running 1 table model in 0 hours 0 minutes and 12.84 seconds (12.84s).
13:38:38
13:38:38 Completed with 1 error and 0 warnings:
13:38:38
13:38:38 (trino.exceptions.TrinoUserError) TrinoUserError(type=USER_ERROR, name=NOT_FOUND, message="Schema lakehouse_gold not found", query_id=20230306_133838_00374_eem4m)
13:38:38 [SQL:
13:38:38 CREATE TABLE lakehouse_gold.__dbt_fal_temp_experimental__experimental (
13:38:38 first_name VARCHAR
13:38:38 )
13:38:38
13:38:38 ]
13:38:38 (Background on this error at: https://sqlalche.me/e/14/f405)
13:38:38
13:38:38 Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1
The issue it that the database
is not prepended to the table FQN (we only have the schema
).