dbt-bigquery-monitoring
dbt-bigquery-monitoring copied to clipboard
Limitation/"Bug": INFORMATION_SCHEMA.PARTITIONS requests are limited to 1000 tables
See https://cloud.google.com/bigquery/docs/information-schema-partitions It leads to failing executions of models on setup with more than 1000 tables
As there's no immediate workaround (maybe through a dedicated materialization like for project one).
In the meantime, if you run into the issue, you can disable the related model by adding in your configuration (dbt_project.yml) :
...
models:
dbt_bigquery_monitoring:
monitoring:
storage:
datamart:
table:
partitions_monitoring:
+enabled: false
intermediate:
stg_partitions_monitoring:
+enabled: false