dbt-bigquery
dbt-bigquery copied to clipboard
[Bug] Cluster order for Materialized Views is mixed up
Is this a new bug in dbt-bigquery?
- [X] I believe this is a new bug in dbt-bigquery
- [X] I have searched the existing issues, and I could not find an existing issue for this bug
Current Behavior
After setting clusters for a materialized view via cluster_by in the Config macro, the order of the clusters is not adopted in the appropriate order as with other table forms, but is mixed according to a pattern that is not apparent.
Expected Behavior
Cluster order is adopted 1:1.
Steps To Reproduce
- Create a model as "materialized_view"
- Set clusters via
cluster_by
parameter - See mixed cluster order in
target
->run
folder or result in BigQuery console
Example model:
{{
config(
materialized="materialized_view",
cluster_by=[ "portal", "cu_name", "rt_name", "ids_in_container"],
)
}}
select * from {{ ref('orders') }}
Result in target folder:
create materialized view if not exists `xxx`
cluster by rt_name, ids_in_container, portal, cu_name
Relevant log output
No response
Environment
- OS: MacOS Sonoma 14.0
- Python: 3.11.1
- dbt-core: 1.7.8
- dbt-bigquery: 1.7.5
Additional Context
No response
Thanks for reporting this @dennishendricks !
I didn't confirm or deny, but I suspect this is involved: https://github.com/dbt-labs/dbt-bigquery/blob/5501cd34b12965e0380952bdbb65fd52f49b49f5/dbt/adapters/bigquery/relation_configs/_cluster.py#L23
Since the cluster key is being converted into an unordered set rather than an ordered list, then the keys aren't guaranteed to preserve the order by the time we reach here.
Hi @dbeatty10, I have created a PR that fixes this bug. Would you mind taking a look at it? Thanks in advance.
I added the triage label to get this back on the Product backlog given the new PR.
Hi @mikealfare, I just updated the PR to the latest changes from main
. The PR is ready for review.