dbt-bigquery-monitoring icon indicating copy to clipboard operation
dbt-bigquery-monitoring copied to clipboard

Question: Workaround to use this package for multiple regions

Open yu-iskw opened this issue 6 months ago • 1 comments

It is quite common to analyze information schemas of BigQuery in multiple regions. As far as I know, dbt doesn't offer to use the same dbt Package multiple times in a dbt Project. However, I want to manage sets of dbt models by dbt-bigquery-monitoring for multiple regions in a single dbt Project. Is there any solution?

yu-iskw avatar May 21 '25 02:05 yu-iskw

There's one approach I'm thinking of that you could leverage: environment variables. Then you can switch the region identifier using DBT_BQ_MONITORING_REGION (see the doc). Would it work for you?

Kayrnt avatar May 21 '25 04:05 Kayrnt

Hi @Kayrnt, thank you for answering my question. We can also dynamically switch the destination project and dataset by environment variables. I am looking for a solution to manage multiple instances of the same dbt Package.

I have looked into several discussions which are similar to what I want to do. However, I haven't found the exactly same. And as far as I know, there are some technical difficulties in name spaces and so on. So, I opened the issue to allow multiple instances of the same dbt Package in a project, because I think it would be useful for other users.

https://github.com/dbt-labs/dbt-core/issues/11647

yu-iskw avatar May 27 '25 22:05 yu-iskw

That's an interesting feature! Let's see how it goes, obviously it would still need to run a profile that set the right location as well but I guess that you would also need to be able to use the dbt variable into the profile.

github-christophe-oudar avatar May 28 '25 20:05 github-christophe-oudar