[Feature] Support the updating of incremental model description after initial creation

[kiwamizamurai]

Is this a new bug in dbt-bigquery?

• [X] I believe this is a new bug in dbt-bigquery
• [X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

Table description does not changed even thought I modify the schema.yml with persist_docs=true and materialized=incremental.

Expected Behavior

Table description should be altered regardless of materialization type.

Steps To Reproduce

1. write the hogehoge.sql as follows

{{
  config(
    materialized = 'incremental',
    persist_docs = {"relation": true, "columns": true}
  )
}}

select 1 as first_col
from unnest([struct(1 as dual)]) as dual

{% if is_incremental() %}
    where 1=1
{% endif %}

2. write the hogehoge.yaml as follows

version: 2

models:
  - name: hogehoge
    description: before_changed
    columns:
      - name: first_col
        description: before_changed

3. run the command dbt build --select hogehoge
4. confirm that the table is created in bigquery console
5. modify the sample.yaml as follows

version: 2

models:
  - name: hogehoge
    description: after_changed
    columns:
      - name: first_col
        description: after_changed

6. confirm that only the column description is updated

Relevant log output

08:36:20  Concurrency: 4 threads (target='dev')
08:36:20  
08:36:20  1 of 1 START sql table model dbt_some_project.hogehoge ...................... [RUN]
08:36:24  1 of 1 OK created sql table model dbt_some_project.hogehoge ................. [CREATE TABLE (1.0 rows, 0 processed) in 4.13s]
08:36:24  
08:36:24  Finished running 1 table model in 0 hours 0 minutes and 9.00 seconds (9.00s).
08:36:24  
08:36:24  Completed successfully

Environment

- OS: MacOS13.6
- Python: 3.11.7
- dbt-core: 1.7.6
- dbt-bigquery: 1.7.6

Additional Context

I have not observed this phenomenon with other data platform adapter like snowflake.

[kiwamizamurai]

the implementation is around here?

https://github.com/dbt-labs/dbt-bigquery/blob/6c0afe4cfb69761dada5d16150fe632b8f72bf39/dbt/include/bigquery/macros/adapters.sql#L48-L52

[dataders]

thanks for opening @kiwamizamurai! Your example was excellent, I was able to reproduce!

AFAICT,column-level docs persistence was first introduced via https://github.com/dbt-labs/dbt-core/pull/2402. The comment above bigquery__persist_docs is telling:

https://github.com/dbt-labs/dbt-bigquery/blob/955afbd74b19d37f6c7919a4a92dd2cc50eae905/dbt/include/bigquery/macros/adapters.sql#L98-L99

From this, I'm assuming the implementation is that a relation (table) description is only created at time of table creation, and not updated like that of columns.

This is a nitpick, but I accordingly changed this from being a bug to an enhancement, as we've never supported in dbt-bigquery the modification of table description metadata after creation.

However, I see no reason why bigquery__persist_docs could not be modified to include an "update table description if different that what is currently documented, similar to the .alter_table_comment() method you propose in #1139. However, I see the adapter already has a update_table_description() method that might be of use instead?

https://github.com/dbt-labs/dbt-bigquery/blob/955afbd74b19d37f6c7919a4a92dd2cc50eae905/dbt/adapters/bigquery/impl.py#L627-L636

[kiwamizamurai]

I understood it. Thank you. @dataders So, I should use .update_table_description() instead of .alter_table_comment() in my PR https://github.com/dbt-labs/dbt-bigquery/pull/1139. Is is right?

https://github.com/kiwamizamurai/dbt-bigquery/blob/fix/1138-table-description/dbt/include/bigquery/macros/adapters.sql#L52-L54

[dataders]

So, I should use .update_table_description() instead of . alter_table_comment() in my PR #1139.

Yeah give it a try? Maybe .update_table_description() doesn't work right and we need to change or delete it, but i think it it worth trying it out.

if that works, then we include a call to it within bigquery__persist_docs

[kiwamizamurai]

@dataders OK, Let me try it.

[kiwamizamurai]

I modified the code https://github.com/dbt-labs/dbt-bigquery/pull/1139/commits/13f17015adbdc2dad4078e01f8f5cf53ad48f693