Add "description" attribute to BQ tables

[kirecek]

Hello,

this PR adds a description argument to big query tables.

I find description quite useful, but sadly it's breaking change becacuse of the nature of terraform objects. This could be avoided using optional values https://www.terraform.io/language/expressions/type-constraints#optional-object-type-attributes but on the other hand It would force users to use terraform >= 1.3 ... so again .. breaking.

So in the end I am not sure if it's worth to have description here but I let you deside :D

[comment-bot-dev]

Thanks for the PR! 🚀 Unfortunately it looks like some of our CI checks failed. See the Contributing Guide for details.

• ⚠️check_documentation The documentation needs to be regenerated. Please run make generate_docs.

Checking for documentation generation
diff -r '--exclude=.terraform' '--exclude=.kitchen' '--exclude=.git' '--exclude=autogen' '--exclude=*.tfvars' /workspace/README.md /tmp/tmp.XyFFhv8FUl/generate_docs/workspace/README.md
195c195
< | external\_tables | A list of objects which include table\_id, expiration\_time, external\_data\_configuration, and labels. | <pre>list(object({<br>    table_id              = string,<br>    autodetect            = bool,<br>    compression           = string,<br>    ignore_unknown_values = bool,<br>    max_bad_records       = number,<br>    schema                = string,<br>    source_format         = string,<br>    source_uris           = list(string),<br>    csv_options = object({<br>      quote                 = string,<br>      allow_jagged_rows     = bool,<br>      allow_quoted_newlines = bool,<br>      encoding              = string,<br>      field_delimiter       = string,<br>      skip_leading_rows     = number,<br>    }),<br>    google_sheets_options = object({<br>      range             = string,<br>      skip_leading_rows = number,<br>    }),<br>    hive_partitioning_options = object({<br>      mode              = string,<br>      source_uri_prefix = string,<br>    }),<br>    expiration_time = string,<br>    labels          = map(string),<br>  }))</pre> | `[]` | no |
---
> | external\_tables | A list of objects which include table\_id, expiration\_time, external\_data\_configuration, and labels. | <pre>list(object({<br>    table_id              = string,<br>    autodetect            = bool,<br>    compression           = string,<br>    ignore_unknown_values = bool,<br>    max_bad_records       = number,<br>    schema                = string,<br>    source_format         = string,<br>    source_uris           = list(string),<br>    csv_options = object({<br>      quote                 = string,<br>      allow_jagged_rows     = bool,<br>      allow_quoted_newlines = bool,<br>      encoding              = string,<br>      field_delimiter       = string,<br>      skip_leading_rows     = number,<br>    }),<br>    google_sheets_options = object({<br>      range             = string,<br>      skip_leading_rows = number,<br>    }),<br>    hive_partitioning_options = object({<br>      mode              = string,<br>      source_uri_prefix = string,<br>    }),<br>    expiration_time = string,<br>    labels          = map(string),<br>    description     = string,<br>  }))</pre> | `[]` | no |
197c197
< | materialized\_views | A list of objects which includes view\_id, view\_query, clustering, time\_partitioning, range\_partitioning, expiration\_time and labels | <pre>list(object({<br>    view_id             = string,<br>    query               = string,<br>    enable_refresh      = bool,<br>    refresh_interval_ms = string,<br>    clustering          = list(string),<br>    time_partitioning = object({<br>      expiration_ms            = string,<br>      field                    = string,<br>      type                     = string,<br>      require_partition_filter = bool,<br>    }),<br>    range_partitioning = object({<br>      field = string,<br>      range = object({<br>        start    = string,<br>        end      = string,<br>        interval = string,<br>      }),<br>    }),<br>    expiration_time = string,<br>    labels          = map(string),<br>  }))</pre> | `[]` | no |
---
> | materialized\_views | A list of objects which includes view\_id, view\_query, clustering, time\_partitioning, range\_partitioning, expiration\_time and labels | <pre>list(object({<br>    view_id             = string,<br>    query               = string,<br>    enable_refresh      = bool,<br>    refresh_interval_ms = string,<br>    clustering          = list(string),<br>    time_partitioning = object({<br>      expiration_ms            = string,<br>      field                    = string,<br>      type                     = string,<br>      require_partition_filter = bool,<br>    }),<br>    range_partitioning = object({<br>      field = string,<br>      range = object({<br>        start    = string,<br>        end      = string,<br>        interval = string,<br>      }),<br>    }),<br>    expiration_time = string,<br>    labels          = map(string),<br>    description     = string,<br>  }))</pre> | `[]` | no |
200,201c200,201
< | tables | A list of objects which include table\_id, schema, clustering, time\_partitioning, range\_partitioning, expiration\_time and labels. | <pre>list(object({<br>    table_id   = string,<br>    schema     = string,<br>    clustering = list(string),<br>    time_partitioning = object({<br>      expiration_ms            = string,<br>      field                    = string,<br>      type                     = string,<br>      require_partition_filter = bool,<br>    }),<br>    range_partitioning = object({<br>      field = string,<br>      range = object({<br>        start    = string,<br>        end      = string,<br>        interval = string,<br>      }),<br>    }),<br>    expiration_time = string,<br>    labels          = map(string),<br>  }))</pre> | `[]` | no |
< | views | A list of objects which include view\_id and view query | <pre>list(object({<br>    view_id        = string,<br>    query          = string,<br>    use_legacy_sql = bool,<br>    labels         = map(string),<br>  }))</pre> | `[]` | no |
---
> | tables | A list of objects which include table\_id, schema, clustering, time\_partitioning, range\_partitioning, expiration\_time and labels. | <pre>list(object({<br>    table_id   = string,<br>    schema     = string,<br>    clustering = list(string),<br>    time_partitioning = object({<br>      expiration_ms            = string,<br>      field                    = string,<br>      type                     = string,<br>      require_partition_filter = bool,<br>    }),<br>    range_partitioning = object({<br>      field = string,<br>      range = object({<br>        start    = string,<br>        end      = string,<br>        interval = string,<br>      }),<br>    }),<br>    expiration_time = string,<br>    labels          = map(string),<br>    description     = string,<br>  }))</pre> | `[]` | no |
> | views | A list of objects which include view\_id and view query | <pre>list(object({<br>    view_id        = string,<br>    query          = string,<br>    use_legacy_sql = bool,<br>    labels         = map(string),<br>    description    = string,<br>  }))</pre> | `[]` | no |
diff -r '--exclude=.terraform' '--exclude=.kitchen' '--exclude=.git' '--exclude=autogen' '--exclude=*.tfvars' /workspace/examples/basic_view/README.md /tmp/tmp.XyFFhv8FUl/generate_docs/workspace/examples/basic_view/README.md
15c15
< | tables | A list of maps that includes table\_id, schema, clustering, time\_partitioning, range\_partitioning, view, expiration\_time, labels in each element. | <pre>list(object({<br>    table_id   = string,<br>    schema     = string,<br>    clustering = list(string),<br>    time_partitioning = object({<br>      expiration_ms            = string,<br>      field                    = string,<br>      type                     = string,<br>      require_partition_filter = bool,<br>    }),<br>    range_partitioning = object({<br>      field = string,<br>      range = object({<br>        start    = string,<br>        end      = string,<br>        interval = string,<br>      }),<br>    }),<br>    expiration_time = string,<br>    labels          = map(string),<br>  }))</pre> | `[]` | no |
---
> | tables | A list of maps that includes table\_id, schema, clustering, time\_partitioning, range\_partitioning, view, expiration\_time, labels in each element. | <pre>list(object({<br>    table_id   = string,<br>    schema     = string,<br>    clustering = list(string),<br>    time_partitioning = object({<br>      expiration_ms            = string,<br>      field                    = string,<br>      type                     = string,<br>      require_partition_filter = bool,<br>    }),<br>    range_partitioning = object({<br>      field = string,<br>      range = object({<br>        start    = string,<br>        end      = string,<br>        interval = string,<br>      }),<br>    }),<br>    expiration_time = string,<br>    labels          = map(string),<br>    description     = string,<br>  }))</pre> | `[]` | no |
18c18
< | views | A list of objects which include table\_id, which is view id, and view query | <pre>list(object({<br>    view_id        = string,<br>    query          = string,<br>    use_legacy_sql = bool,<br>    labels         = map(string),<br>  }))</pre> | `[]` | no |
---
> | views | A list of objects which include table\_id, which is view id, and view query | <pre>list(object({<br>    view_id        = string,<br>    query          = string,<br>    use_legacy_sql = bool,<br>    labels         = map(string),<br>    description    = string,<br>  }))</pre> | `[]` | no |
Error: Documentation generation has not been run, please run the
'make docker_generate_docs' command and commit the above changes.

• ⚠️check_terraform Failed Terraform check. More details below.

Running terraform fmt
Running terraform validate
terraform_validate . 
Success!
The configuration is valid.
terraform_validate ./examples/basic_bq 
Success!
The configuration is valid.
terraform_validate ./examples/basic_view 
Success!
The configuration is valid.
terraform_validate ./examples/multiple_tables 
╷
│

Error: 
Invalid value for module argument
│

│

 on main.tf line 26, in module "bigquery":
│

 26:   tables = 
[
│

 27:     {
│

 28:       table_id = "foo",
│

 29:       schema   = file("sample_bq_schema.json"),
│

 30:       time_partitioning = {
│

 31:         type                     = "DAY",
│

 32:         field                    = null,
│

 33:         require_partition_filter = false,
│

 34:         expiration_ms            = null,
│

 35:       },
│

 36:       description        = "some description",
│

 37:       range_partitioning = null,
│

 38:       expiration_time    = null,
│

 39:       clustering         = ["fullVisitorId", "visitId"],
│

 40:       labels = {
│

 41:         env      = "dev"
│

 42:         billable = "true"
│

 43:         owner    = "joedoe"
│

 44:       },
│

 45:     },
│

 46:     {
│

 47:       table_id          = "bar",
│

 48:       schema            = file("sample_bq_schema.json"),
│

 49:       time_partitioning = null,
│

 50:       range_partitioning = {
│

 51:         field = "visitNumber",
│

 52:         range = {
│

 53:           start    = "1"
│

 54:           end      = "100",
│

 55:           interval = "10",
│

 56:         },
│

 57:       },
│

 58:       expiration_time = 2524604400000, # 2050/01/01
│

 59:       clustering      = [],
│

 60:       labels = {
│

 61:         env      = "devops"
│

 62:         billable = "true"
│

 63:         owner    = "joedoe"
│

 64:       },
│

 65:     }
│

 66:   ]
│

│

The given value is not suitable for child module variable "tables" defined
│

at ../../variables.tf:91,1-18: element 1: attribute "description" is
│

required.
╵
terraform_validate ./examples/scheduled_queries 
Success!
The configuration is valid.
terraform_validate ./modules/authorization 
Success!
The configuration is valid.
terraform_validate ./modules/scheduled_queries 
Success!
The configuration is valid.
terraform_validate ./modules/udf 
Success!
The configuration is valid.
terraform_validate ./test/fixtures/full 
╷
│

Error: 
Invalid value for module argument
│

│

 on ../../../examples/multiple_tables/main.tf line 26, in module "bigquery":
│

 26:   tables = 
[
│

 27:     {
│

 28:       table_id = "foo",
│

 29:       schema   = file("sample_bq_schema.json"),
│

 30:       time_partitioning = {
│

 31:         type                     = "DAY",
│

 32:         field                    = null,
│

 33:         require_partition_filter = false,
│

 34:         expiration_ms            = null,
│

 35:       },
│

 36:       description        = "some description",
│

 37:       range_partitioning = null,
│

 38:       expiration_time    = null,
│

 39:       clustering         = ["fullVisitorId", "visitId"],
│

 40:       labels = {
│

 41:         env      = "dev"
│

 42:         billable = "true"
│

 43:         owner    = "joedoe"
│

 44:       },
│

 45:     },
│

 46:     {
│

 47:       table_id          = "bar",
│

 48:       schema            = file("sample_bq_schema.json"),
│

 49:       time_partitioning = null,
│

 50:       range_partitioning = {
│

 51:         field = "visitNumber",
│

 52:         range = {
│

 53:           start    = "1"
│

 54:           end      = "100",
│

 55:           interval = "10",
│

 56:         },
│

 57:       },
│

 58:       expiration_time = 2524604400000, # 2050/01/01
│

 59:       clustering      = [],
│

 60:       labels = {
│

 61:         env      = "devops"
│

 62:         billable = "true"
│

 63:         owner    = "joedoe"
│

 64:       },
│

 65:     }
│

 66:   ]
│

│

The given value is not suitable for child module variable "tables" defined
│

at ../../../variables.tf:91,1-18: element 1: attribute "description" is
│

required.
╵
terraform_validate ./test/setup 
Success!
The configuration is valid.

[github-actions[bot]]

This PR is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days