Kayrnt
Database Error in model information_schema_jobs (models\information_schema\jobs\information_schema_jobs.sql)
Database Error in model information_schema_jobs (models\information_schema\jobs\information_schema_jobs.sql) Invalid cast from STRUCT<index_usage_mode STRING, index_unused_reasons ARRAY<STRUCT<code STRING, message STRING, base_table STRUCT<project_id STRING, dataset_id STRING, table_id STRING>, ...>>, index_pruning_stats ARRAY<STRUCT<base_table STRUCT<project_id STRING, dataset_id STRING, table_id STRING>, pre_index_pruning_parallel_input_count INT64, post_index_pruning_parallel_input_count INT64>>> to STRUCT<index_usage_mode STRING, index_unused_reasons ARRAY<STRUCT<code STRING, message STRING, base_table STRUCT<project_id STRING, dataset_id STRING, table_id STRING>, ...>>, index_pruning_stats ARRAY<STRUCT<base_table STRUCT<project_id STRING, dataset_id STRING, table_id STRING>, pre_index_pruning_parallel_input_count INT64, post_index_pruning_parallel_input_count INT64, ...>>> at [50:16]
FYI am using the latest version
- package: bqbooster/dbt_bigquery_monitoring version: ["0.23.0"]
Hello, thanks for reporting, I guess the schema from that field was updated and dbt... doesn't support automatic schema evolution on STRUCT. I'll make the update and try to find a way to make it more resilient.
I had another thought about that problem: it didn't add a new field but it feels like Google updated the schema and the problem is that I can't really guard about their deployment (especially without dbt supporting schema evolution on the STRUCT). The best I might do is to try to generate the schema evolution script (so far the single solution I found what to dump schema using bq CLI, update the struct and push the update). I might be able to to provide a Python package to do it if that makes sense for you?
FYI I'm preparing a dbt bigquery change to support STRUCT field evolution. In the meantime 2 solutions for you:
- full refresh the model (destroying the history)
- updating manually the schema (see my blogpost about it)