Configuration drift due to settings.version in the outputs of the modules

[verdel]

TL;DR

In the resources of google_sql_database_instance, there is an attribute settings.version that cannot be specified or modified in the configuration, and it is set externally. Due to the fact that complete google_sql_database_instance resources are constantly passed in the output of modules, configuration drift occurs.

Expected behavior

We do not modify the configuration in the Terraform code, and when we run terraform plan, we receive information that everything is in the up-to-date state

Observed behavior

As the value of the settings.version attribute can change externally, when we run terraform plan, we receive a message about configuration drift.

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the
last "terraform apply" which may have affected this plan:

  # google_sql_database_instance.default has changed
  ~ resource "google_sql_database_instance" "default" {
        id                             = "test"
        name                           = "test"
        # (14 unchanged attributes hidden)

      ~ settings {
          ~ version                     = 74 -> 76
            # (11 unchanged attributes hidden)

            # (6 unchanged blocks hidden)
        }

        # (1 unchanged block hidden)
    }


Unless you have made equivalent changes to your configuration, or ignored the
relevant attributes using ignore_changes, the following plan may include
actions to undo or respond to these changes.

─────────────────────────────────────────────────────────────────────────────

Changes to Outputs:
  ~ instances                                         = (sensitive value)
  ~ primary                                           = (sensitive value)

Terraform Configuration

module "sql-db" {
  source  = "GoogleCloudPlatform/sql-db/google//modules/postgresql"
  version = "8.0.0"
  zone              = "${local.region}-c"
  availability_type = "ZONAL"

  maintenance_window_day          = 7
  maintenance_window_hour         = 12
  maintenance_window_update_track = "stable"

  deletion_protection = true

  ip_configuration = {
    ipv4_enabled        = false
    private_network     = vpc.outputs.network_id
    allocated_ip_range  = psa.google_compute_global_address_name
    require_ssl         = true
    authorized_networks = []
  }

  database_flags = [
    {
      name  = "cloudsql.iam_authentication"
      value = "on"
    },
  ]

  enable_default_db    = false
  enable_default_user  = true
  user_name            = "backup"
  user_password        = ""
  additional_databases = []
  additional_users     = []
  iam_user_emails = [
    "[email protected]",
    "[email protected]"
  ]

  create_timeout = "30m"
  update_timeout = "30m"
  delete_timeout = "30m"

  backup_configuration = {
    enabled                        = true
    point_in_time_recovery_enabled = true
    start_time                     = "00:00"
    transaction_log_retention_days = "7"
    retained_backups               = 7
    retention_unit                 = "COUNT"
    location                       = "eu"
  }
}

Terraform Version

Terraform v1.6.5
on darwin_arm64
+ provider registry.terraform.io/hashicorp/google v4.84.0
+ provider registry.terraform.io/hashicorp/google-beta v4.84.0
+ provider registry.terraform.io/hashicorp/null v3.2.2
+ provider registry.terraform.io/hashicorp/random v3.5.1

Additional information

As a solution, we could filter out settings.version attribute in the output variables as follows:

locals {
  primary_output = { for k, v in google_sql_database_instance.default : k => k == "settings" ? toset([{for key, value in v[0] : key => value if key != "version" }]) : v }
  replicas_output = {for name, replica in google_sql_database_instance.replicas: name => { for k, v in replica : k => k == "settings" ? toset([{for key, value in v[0] : key => value if key != "version" }]) : v }}
}

// Resources
output "primary" {
  value       = local.primary_output
  description = "The `google_sql_database_instance` resource representing the primary instance"
  sensitive   = true
}

output "replicas" {
  value       = values(local.replicas_output)
  description = "A list of `google_sql_database_instance` resources representing the replicas"
  sensitive   = true
}

output "instances" {
  value       = concat([local.primary_output], values(local.replicas_output))
  description = "A list of all `google_sql_database_instance` resources we've created"
  sensitive   = true
}

Changes in the code have been made using the example of the PostgreSQL module. If this solution works for you, I could prepare a pull request and update the output variables in all affected modules.

[imrannayer]

@verdel this seems like provider issue. Is it possible if you can recreate this using resource directly and create an issue for the provider team here.

[verdel]

Perhaps, I wasn't able to fully describe the issue. The settings.version attribute is always set externally and cannot be modified through Terraform configurations. Because the entire SQL instance resource is passed in the output, any change in settings.version results in a non-empty plan. If the resource were not passed in the output, or if the settings.version attribute were removed from it during output, the plan would remain empty even when modifying this attribute.

Are you absolutely certain that the described issue is an upstream problem?

[imrannayer]

@verdel yes this is something provider can handle. Just let them know this is a readonly setting and should not be shown as diff. Once you have the issue created can you mention the link here so we get an update.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days

[ggorge-etiqa]

Why don't just use lifecycle ignore_changes in the module?

  lifecycle {
    ignore_changes = [
      settings["version"],
    ]
  }

[verdel]

Why don't just use lifecycle ignore_changes in the module?

  lifecycle {
    ignore_changes = [
      settings["version"],
    ]
  }

As far as I know, ignore_changes only affects ignoring changes in specific attributes of a resource when using terraform plan and terraform apply. If the entire resource or the ignored attribute is used as a value in an output variable, then the actual value of the attribute or all attributes of the resource will still be calculated, and in the end, we will get this behavior:

module.wrapper.custom_resource.foo: Refreshing state... [id=./terraform-provider-custom_resource_test]

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply" which may have affected this plan:

  # module.wrapper.custom_resource.foo has changed
  ~ resource "custom_resource" "foo" {
        id             = "./terraform-provider-custom_resource_test"
      ~ state          = "qwe" -> "qwe1"
        # (6 unchanged attributes hidden)
    }


Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes, the following plan may include actions to undo or respond to these changes.

Changes to Outputs:
  ~ wrapper = "qwe" -> "qwe1"

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

So, using ignore_changes in the code of this module for the resource google_sql_database_instance will not solve the problem.

The resolution to this issue could be achieved by either omitting the attribute settings.version from the outputs, or by adapting the provider's logic to ensure that modifications to the settings.version attribute do not trigger updates to the google_sql_database_instance resource.

[imrannayer]

@verdel do you have an issue open for the provider? If so can u mention it here so we can track it?

[verdel]

I apologize, but due to the workload, I haven't been able to find time for this. Now I've returned to this issue and will try to reproduce the error again to gather data for opening new issue in the provider repository

[verdel]

I haven't been able to reproduce the problem yet. At some point, we stopped using the terraform-google-sql-db module and now work directly with the google_sql_database_instance resource.

A few days ago, I specifically created such a resource and outputted the settings value in a test module to capture the moment when no changes were made to the settings by the terraform code, but the settings.version value changed.

So far, this hasn't happened. If someone could help me with preparing the data for opening an issue in the provider's repository, I would be very grateful.

I need to confirm that the value of settings.version can indeed change, while all other fields in settings remain unchanged.

[verdel]

At the moment, I have managed to reproduce the change of settings.version only with the maintenance_version attribute change, which Google makes from time to time as part of standard maintenance procedures. This is a valid state change, and settings.version does not change separately from another attribute.

[imrannayer]

@verdel provider should not show the drift on parameters which we dont have any control over. For that reason they need to ignore these changes for diff. They can still show it just not as diff.

[verdel]

@imrannayer, I apologize for taking your time to discuss potentially elementary questions. I created this issue because I was experiencing configuration drift due to the change in the settings.version attribute even though I had not made any changes to the Cloud SQL instance settings and there were no such changes on the Google Cloud side.

At the moment, I was only able to capture the change in the settings.version attribute once, and at the same time, the maintenance_version attribute changed as well, since Google Cloud automatically updates the used version of PostgreSQL. Therefore, I have not been able to reproduce the problem described in the issue.

I can achieve the behavior described in the issue by directly changing the version value in the tfstate file and running terraform plan. It turns out, I cannot assert with certainty that the change in the settings.version attribute can occur without altering any other attribute.

If the issue is confirmed, am I correct in understanding that I would need to suggest to the provider development team to add a DiffSuppressFunc function to the version attribute schema description, which would return false if none of the remaining attributes have been changed?

Or did you imply a different solution to the problem with the attribute:

• This attribute should be removed from the schema
• The DiffSuppressFunc function should always return false

Is there possibly another way to indicate that an attribute change should not be displayed as a resource diff?

In any case, I need to wait for the problem to be reproduced. Without confirmation, I see no point in opening an issue in the provider's upstream repository.

[juliusoh]

was there a solution for this?

[verdel]

@juliusoh, Is it possible for you to provide me with the output of terraform plan that shows a diff solely for the settings.version parameter, without any other parameters changing (such as the database version in use)?

[juliusoh]

@juliusoh, Is it possible for you to provide me with the output of terraform plan that shows a diff solely for the settings.version parameter, without any other parameters changing (such as the database version in use)?

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:

• create

Terraform will perform the following actions:

module.pg.google_sql_database.default[0] will be created

• resource "google_sql_database" "default" {
  • charset = "UTF8"
  • collation = "en_US.UTF8"
  • deletion_policy = "DELETE"
  • id = (known after apply)
  • instance = "******-stage-db-02df5265"
  • name = "******-stage-db"
  • project = "******-stage"
  • self_link = (known after apply) }

Plan: 1 to add, 0 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.

module.pg.google_sql_database_instance.default has changed~ resource "google_sql_database_instance" "default" { id = "builderfax-stage-db-02df5265" name = "builderfax-stage-db-02df5265" # (14 unchanged attributes hidden)~ settings { ~ version = 18 -> 21 # (12 unchanged attributes hidden)

    # (5 unchanged blocks hidden)
}

# (1 unchanged block hidden)

}Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes, the following plan may include actions to undo or respond to these changes

[verdel]

@juliusoh, thank you very much for the information. I will try to open an Issue in the terraform provider repository as soon as possible.

[verdel]

@juliusoh, @imrannayer, I created an issue in the provider's repository.