terraform-provider-google icon indicating copy to clipboard operation
terraform-provider-google copied to clipboard

google_monitoring_dashboard: resource still has perma diffs

Open IchordeDionysos opened this issue 2 years ago • 10 comments

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
  • Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment.
  • If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

I hope this helps reproduce the issue and I hope we don't need to wait until the next major version again to get this fixed 😅

This is a long-standing annoyance, polluting the changes when applying Terraform, leading to important differences needing to be noticed more easily.

Terraform Version

v1.6.0

Affected Resource(s)

  • google_monitoring_dashboard

Terraform Configuration Files

Full Example reproducing the issue:

terraform {
  required_version = ">= 1.3.0, < 2.0.0"

  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 5.0"
    }
  }
}

variable "project" {
  type = string
}

variable "region" {
  type = string
}

provider "google" {
  project = var.project
  region  = var.region
}

resource "google_monitoring_dashboard" "dashboard" {

  dashboard_json = <<EOF
  {
  "dashboardFilters": [],
  "displayName": "Perma diff dashboard",
  "labels": {},
  "mosaicLayout": {
    "columns": 12,
    "tiles": [
      {
        "height": 4,
        "widget": {
          "title": "SLO Compliance: 99.9% - Good/Total Ratio - Rolling 28 days",
          "xyChart": {
            "chartOptions": {
              "mode": "COLOR"
            },
            "dataSets": [
              {
                "breakdowns": [],
                "dimensions": [],
                "measures": [],
                "plotType": "LINE",
                "targetAxis": "Y1",
                "timeSeriesQuery": {
                  "timeSeriesFilter": {
                    "aggregation": {
                      "perSeriesAligner": "ALIGN_NEXT_OLDER"
                    },
                    "filter": "select_slo_compliance(\"projects/${var.project}/services/run-service/serviceLevelObjectives/run-service-rolling-availability-slo\")"
                  },
                  "unitOverride": "10^2.%"
                }
              }
            ],
            "thresholds": [
              {
                "label": "",
                "targetAxis": "Y1",
                "value": 0.999
              }
            ]
          }
        },
        "width": 12
      },
      {
        "height": 4,
        "widget": {
          "title": "SLO Compliance: 95% - 90th percentile latency under 1000ms - Rolling 28 days",
          "xyChart": {
            "chartOptions": {
              "mode": "COLOR"
            },
            "dataSets": [
              {
                "breakdowns": [],
                "dimensions": [],
                "measures": [],
                "plotType": "LINE",
                "targetAxis": "Y1",
                "timeSeriesQuery": {
                  "timeSeriesFilter": {
                    "aggregation": {
                      "perSeriesAligner": "ALIGN_NEXT_OLDER"
                    },
                    "filter": "select_slo_compliance(\"projects/${var.project}/services/run-service/serviceLevelObjectives/run-service-latency-slo\")"
                  },
                  "unitOverride": "10^2.%"
                }
              }
            ],
            "thresholds": [
              {
                "label": "",
                "targetAxis": "Y1",
                "value": 0.95
              }
            ]
          }
        },
        "width": 12,
        "yPos": 4
      },
      {
        "height": 4,
        "widget": {
          "title": "Requests per response code class",
          "xyChart": {
            "chartOptions": {
              "mode": "COLOR"
            },
            "dataSets": [
              {
                "breakdowns": [],
                "dimensions": [],
                "measures": [],
                "minAlignmentPeriod": "60s",
                "plotType": "LINE",
                "targetAxis": "Y1",
                "timeSeriesQuery": {
                  "timeSeriesFilter": {
                    "aggregation": {
                      "alignmentPeriod": "60s",
                      "perSeriesAligner": "ALIGN_RATE"
                    },
                    "filter": "metric.type=\"loadbalancing.googleapis.com/https/request_count\" resource.type=\"https_lb_rule\" (resource.label.backend_target_name=\"api-lb-backend-run-service\")",
                    "secondaryAggregation": {
                      "alignmentPeriod": "60s",
                      "crossSeriesReducer": "REDUCE_MEAN",
                      "groupByFields": [
                        "metric.label.\"response_code_class\""
                      ],
                      "perSeriesAligner": "ALIGN_MEAN"
                    }
                  }
                }
              }
            ],
            "thresholds": [],
            "timeshiftDuration": "0s",
            "yAxis": {
              "label": "y1Axis",
              "scale": "LINEAR"
            }
          }
        },
        "width": 12,
        "yPos": 8
      },
      {
        "height": 4,
        "widget": {
          "title": "Latency 50th & 95th percentile",
          "xyChart": {
            "chartOptions": {
              "mode": "COLOR"
            },
            "dataSets": [
              {
                "breakdowns": [],
                "dimensions": [],
                "measures": [],
                "minAlignmentPeriod": "60s",
                "plotType": "LINE",
                "targetAxis": "Y1",
                "timeSeriesQuery": {
                  "timeSeriesFilter": {
                    "aggregation": {
                      "alignmentPeriod": "60s",
                      "crossSeriesReducer": "REDUCE_PERCENTILE_50",
                      "groupByFields": [],
                      "perSeriesAligner": "ALIGN_DELTA"
                    },
                    "filter": "metric.type=\"loadbalancing.googleapis.com/https/total_latencies\" resource.type=\"https_lb_rule\" (resource.label.backend_target_name=\"api-lb-backend-run-service\")"
                  }
                }
              },
              {
                "breakdowns": [],
                "dimensions": [],
                "measures": [],
                "minAlignmentPeriod": "60s",
                "plotType": "LINE",
                "targetAxis": "Y1",
                "timeSeriesQuery": {
                  "timeSeriesFilter": {
                    "aggregation": {
                      "alignmentPeriod": "60s",
                      "crossSeriesReducer": "REDUCE_PERCENTILE_95",
                      "groupByFields": [],
                      "perSeriesAligner": "ALIGN_DELTA"
                    },
                    "filter": "metric.type=\"loadbalancing.googleapis.com/https/total_latencies\" resource.type=\"https_lb_rule\" (resource.label.backend_target_name=\"api-lb-backend-run-service\")"
                  }
                }
              }
            ],
            "thresholds": [],
            "timeshiftDuration": "0s",
            "yAxis": {
              "label": "y1Axis",
              "scale": "LINEAR"
            }
          }
        },
        "width": 12,
        "yPos": 12
      },
      {
        "height": 4,
        "widget": {
          "title": "Cache Hit Ratio",
          "xyChart": {
            "chartOptions": {
              "mode": "COLOR"
            },
            "dataSets": [
              {
                "breakdowns": [],
                "dimensions": [],
                "measures": [],
                "plotType": "LINE",
                "targetAxis": "Y1",
                "timeSeriesQuery": {
                  "timeSeriesFilterRatio": {
                    "denominator": {
                      "aggregation": {
                        "alignmentPeriod": "60s",
                        "crossSeriesReducer": "REDUCE_SUM",
                        "groupByFields": [
                          "resource.label.\"backend_target_name\""
                        ],
                        "perSeriesAligner": "ALIGN_RATE"
                      },
                      "filter": "metric.type=\"loadbalancing.googleapis.com/https/backend_request_count\" resource.type=\"https_lb_rule\" (resource.label.backend_target_name=\"api-lb-backend-run-service\")"
                    },
                    "numerator": {
                      "aggregation": {
                        "alignmentPeriod": "60s",
                        "crossSeriesReducer": "REDUCE_SUM",
                        "groupByFields": [],
                        "perSeriesAligner": "ALIGN_RATE"
                      },
                      "filter": "metric.type=\"loadbalancing.googleapis.com/https/backend_request_count\" resource.type=\"https_lb_rule\" metric.label.\"cache_result\"=\"HIT\" (resource.label.backend_target_name=\"api-lb-backend-run-service\")"
                    }
                  }
                }
              }
            ],
            "thresholds": [],
            "timeshiftDuration": "0s",
            "yAxis": {
              "label": "y1Axis",
              "scale": "LINEAR"
            }
          }
        },
        "width": 12,
        "yPos": 16
      },
      {
        "height": 7,
        "widget": {
          "logsPanel": {
            "filter": "resource.type=\"cloud_function\"\nlog_name=\"projects/${var.project}/logs/cloudaudit.googleapis.com%2Factivity\"\nprotoPayload.request.function.labels.domain=\"run-service\"",
            "resourceNames": [
              "projects/${var.project}"
            ]
          },
          "title": "API deployments"
        },
        "width": 12,
        "yPos": 20
      }
    ]
  }
}
EOF
}

Minimal example reproducing the issue

terraform {
  required_version = ">= 1.3.0, < 2.0.0"

  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 5.0"
    }
  }
}

variable "project" {
  type    = string
  default = "simpleclub-stage"
}

variable "region" {
  type    = string
  default = "europe-west1"
}

provider "google" {
  project = var.project
  region  = var.region
}

resource "google_monitoring_dashboard" "dashboard" {

  dashboard_json = <<EOF
  {
  "dashboardFilters": [],
  "displayName": "Perma diff dashboard",
  "labels": {},
  "mosaicLayout": {
    "columns": 12,
    "tiles": [
      {
        "height": 4,
        "widget": {
          "title": "SLO Compliance: 99.9% - Good/Total Ratio - Rolling 28 days",
          "xyChart": {
            "chartOptions": {
              "mode": "COLOR"
            },
            "dataSets": [
              {
                "breakdowns": [],
                "dimensions": [],
                "measures": [],
                "plotType": "LINE",
                "targetAxis": "Y1",
                "timeSeriesQuery": {
                  "timeSeriesFilter": {
                    "aggregation": {
                      "perSeriesAligner": "ALIGN_NEXT_OLDER"
                    },
                    "filter": "select_slo_compliance(\"projects/${var.project}/services/run-service/serviceLevelObjectives/run-service-rolling-availability-slo\")"
                  },
                  "unitOverride": "10^2.%"
                }
              }
            ],
            "thresholds": [
              {
                "label": "",
                "targetAxis": "Y1",
                "value": 0.999
              }
            ]
          }
        },
        "width": 12
      }
    ]
  }
}
EOF
}
Terraform output after re-planning (with perma diffs):


Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # google_monitoring_dashboard.dashboard will be updated in-place
  ~ resource "google_monitoring_dashboard" "dashboard" {
      ~ dashboard_json = jsonencode(
          ~ {
              + dashboardFilters = []
              - etag             = "dashboard-etag"
              + labels           = {}
              ~ mosaicLayout     = {
                  ~ tiles   = [
                      ~ {
                          ~ widget = {
                              ~ xyChart = {
                                  ~ dataSets     = [
                                      ~ {
                                          + breakdowns      = []
                                          + dimensions      = []
                                          + measures        = []
                                            # (3 unchanged attributes hidden)
                                        },
                                    ]
                                  ~ thresholds   = [
                                      ~ {
                                          + label      = ""
                                            # (2 unchanged attributes hidden)
                                        },
                                    ]
                                    # (1 unchanged attribute hidden)
                                }
                                # (1 unchanged attribute hidden)
                            }
                            # (2 unchanged attributes hidden)
                        },
                    ]
                    # (1 unchanged attribute hidden)
                }
              - name             = "projects/projectA/dashboards/dashboard-id"
                # (1 unchanged attribute hidden)
            }
        )
        id             = "projects/projectA/dashboards/dashboard-id"
        # (1 unchanged attribute hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.

Debug Output

Expected Behavior

Re-planning shows "No changes".

Actual Behavior

Re-planning shows changes needed for Terraform.

Steps to Reproduce

  1. terraform apply
  2. terraform plan

Important Factoids

It seems like using the mosaicLayout or some specific fields leads to this issue, but I'm not sure if that's indeed the case.

References

  • #7242
  • GoogleCloudPlatform/magic-modules#9197

b/304483210

IchordeDionysos avatar Oct 09 '23 07:10 IchordeDionysos

@c2thorn here is the dedicated issue

IchordeDionysos avatar Oct 09 '23 07:10 IchordeDionysos

@c2thorn did you have time to look into this?

IchordeDionysos avatar Oct 20 '23 13:10 IchordeDionysos

We face the same issue. Would be really helpful to get this resolved!

philipp-wahala-db avatar Nov 14 '23 08:11 philipp-wahala-db

same issue, please fix

devshree-cohere avatar Feb 22 '24 17:02 devshree-cohere

still an issue in 1.7.0 and provider v5.18.0

pjohnston avatar Mar 01 '24 19:03 pjohnston

Hello! This issue still persists - tf version 1.7.0, provider v.5.20.0 Could someone please take a look over it? Many thanks in advance!

alexandra-buica avatar Mar 18 '24 16:03 alexandra-buica

The issue is still presists. Tested in Terraform Version 1.8.2 and 5.26.0 for google terraform provider.

tm185187 avatar Apr 30 '24 12:04 tm185187

Given no progress have been made on this, we switched from mosaicLayout to row layout and were able to get rid of the issues.

Also removing fields that are defaults helped.

This shouldn't be necessary though...

IchordeDionysos avatar May 03 '24 06:05 IchordeDionysos

++ this still exists

rdiers avatar May 17 '24 20:05 rdiers