dbt-databricks
dbt-databricks copied to clipboard
SQL compilation error when running `--empty` flag on on model that utilizes `dbt_utils.union_relations()` macro
trafficstars
Describe the bug
I got an error when trying to use the --empty flag with a model that uses the dbt_utils.union_relations macro.
Workaround
Doing this update seemed to resolve the issue for me (just changing relation|lower to be relation.render()|lower):
{% macro get_columns_comments(relation) -%}
{% call statement('get_columns_comments', fetch_result=True) -%}
describe table {{ relation.render()|lower }}
{% endcall %}
{% do return(load_result('get_columns_comments').table) %}
{% endmacro %}
Steps To Reproduce
Create these files:
models/my_model.sql
select 1 as id
models/my_model_unioned.sql
with unioned as (
{{ dbt_utils.union_relations(
relations=[ref("my_model")]
) }}
)
select * from unioned
Run these commands:
dbt run -s +my_model_unioned --empty
Expected behavior
Completed successfully
Screenshots and log output
Actual log output:
$ dbt run -s +my_model_unioned --empty --profile databricks
16:47:05 Running with dbt=1.8.5
16:47:33 Registered adapter: databricks=1.8.5
16:47:33 Found 2 models, 708 macros
16:47:33
16:47:36 Concurrency: 10 threads (target='databricks')
16:47:36
16:47:36 1 of 2 START sql view model dbt_dbeatty.my_model ............................... [RUN]
16:47:38 1 of 2 OK created sql view model dbt_dbeatty.my_model .......................... [OK in 1.31s]
16:47:38 2 of 2 START sql view model dbt_dbeatty.my_model_unioned ....................... [RUN]
16:47:39 2 of 2 ERROR creating sql view model dbt_dbeatty.my_model_unioned .............. [ERROR in 0.97s]
16:47:40
16:47:40 Finished running 2 view models in 0 hours 0 minutes and 6.54 seconds (6.54s).
16:47:40
16:47:40 Completed with 1 error and 0 warnings:
16:47:40
16:47:40 Runtime Error in model my_model_unioned (models/my_model_unioned.sql)
Runtime Error
[PARSE_SYNTAX_ERROR] Syntax error at or near '('. SQLSTATE: 42601 (line 2, pos 15)
== SQL ==
/* {"app": "dbt", "dbt_version": "1.8.5", "dbt_databricks_version": "1.8.5", "databricks_sql_connector_version": "3.1.2", "profile_name": "databricks", "target_name": "databricks", "node_id": "model.my_project.my_model_unioned"} */
describe table (select * from `hive_metastore`.`dbt_dbeatty`.`my_model` where false limit 0) _dbt_limit_subq_my_model
---------------^^^
16:47:40
16:47:40 Done. PASS=1 WARN=0 ERROR=1 SKIP=0 TOTAL=2
System information
The output of dbt --version:
$ dbt --version
Core:
- installed: 1.8.5
Plugins:
- databricks: 1.8.5 - Update available!
The operating system you're using: N/A
The output of python --version: N/A
Additional context
See https://github.com/dbt-labs/dbt-core/issues/10766#issuecomment-2374593273 for background.