[ADAP-464] Materialized Views - Non-incremental support

[Fleid]

As commented in the core MV thread, BigQuery is adding support for non-incremental MVs.

From our side, it means that we have an additional setting to support for MVs in BQ: allow_non_incremental_definition: true/false. This setting is covered by on_configuration_change via DROP/CREATE since it can't be changed via ALTER.

When set, it requires max_staleness to be set. I'd rather leave BQ errors out here for now - this is a preview feature and I'm not sure this won't change before GA. We should not check this.

---

BigQuery

BigQuery Materialized Views

• Configurations:
  • enable_refresh : BOOLEAN, optional (database level default true) (see lifecycle for effect)
  • refresh_interval_minutes : integer, optional (database level default at 30), only valid for enable_refresh true or default
  • max_staleness : time before bypassing the MV and running the statement instead (in preview but I like it!)
  • allow_non_incremental_definition : BOOLEAN, optional (database default false)
  • From models: Partition / Cluster, TTL, Labels
• Lifecycle : default except Refresh is bypassed if auto_refresh
  • on_configuration_change.apply : enable_refresh/refresh_interval_minutes/max_staleness via ALTER, rest via DROP/CREATE

[Fleid]

@mikealfare FYI

[bnaul]

@Fleid @mikealfare this has been out there for a while now, I think it's basically impossible at this point that they would drastically change the functionality, as they have written it up and pulled it into official client libraries. ~Happy to try my hand at a PR for adding this if it would be welcome!~ Changing to an "ask for forgiveness instead of permission" approach https://github.com/dbt-labs/dbt-bigquery/pull/1011

[LuigiCerone]

Hello, any update on this? :)