Inconsistent ordering of layers in schedules causes unnecessary changes

[mcouthon]

Hi there, we believe that inconsistent ordering of layers in PD schedules causes unnecessary changes in TF runs. This happens sporadically, and we haven't been able to reproduce it consistently, but we've certainly seen this occur when we haven't touched the relevant code (and thus, no change should've occurred).

Terraform Version

0.14.10

Affected Resource(s)

• pagerduty_schedule

Terraform Configuration Files

resource "pagerduty_schedule" "schedule" {
  name      = var.schedule_name
  time_zone = "Asia/Jerusalem"

  layer {
    name                         = "Day on-call"
    rotation_turn_length_seconds = var.schedule_policy == "2d" ? 7 * 24 * 60 * 60 : 24 * 60 * 60
    rotation_virtual_start       = "2020-05-04T18:00:00+03:00"
    start                        = "2020-05-04T18:00:00+03:00"
    users                        = var.members

    dynamic "restriction" {
      for_each = var.schedule_policy == "2d" ? [1] : var.weekly_restriction_days
      content {
        start_day_of_week = restriction.value
        duration_seconds  = var.schedule_policy == "2d" ? 48 * 60 * 60 : 9 * 60 * 60
        start_time_of_day = "09:00:00"
        type              = "weekly_restriction"
      }
    }
  }

  layer {
    name                         = "Night on-call"
    rotation_turn_length_seconds = var.schedule_policy == "2d" ? 7 * 24 * 60 * 60 : 24 * 60 * 60
    rotation_virtual_start       = "2020-05-06T18:00:00+03:00"
    start                        = "2020-05-06T18:00:00+03:00"
    users = concat(
      slice(var.members, 1, length(var.members)),
      slice(var.members, 0, 1)
    )

    dynamic "restriction" {
      for_each = var.schedule_policy == "2d" ? [3] : var.weekly_restriction_days
      content {
        start_day_of_week = restriction.value
        duration_seconds  = var.schedule_policy == "2d" ? 48 * 60 * 60 : 15 * 60 * 60
        start_time_of_day = var.schedule_policy == "2d" ? "09:00:00" : "18:00:00"
        type              = "weekly_restriction"
      }
    }
  }

  layer {
    name                         = "Weekend on-call"
    rotation_turn_length_seconds = 7 * 24 * 60 * 60
    rotation_virtual_start       = "2020-05-08T09:00:00+03:00"
    start                        = "2020-05-08T09:00:00+03:00"
    users = concat(
      slice(var.members, 2, length(var.members)),
      slice(var.members, 0, 2)
    )

    restriction {
      duration_seconds  = var.schedule_policy == "2d" ? 72 * 60 * 60 : 48 * 60 * 60
      start_time_of_day = "09:00:00"
      type              = "weekly_restriction"
      start_day_of_week = 5
    }
  }

  lifecycle {
    ignore_changes = [
      layer[0].start,
      layer[1].start,
      layer[2].start,
      layer[0].rotation_virtual_start,
      layer[1].rotation_virtual_start,
      layer[2].rotation_virtual_start
    ]
  }
}

Expected Behavior

Nothing :)

Actual Behavior

The order of some of the layers got mixed up and reflected in the changes:

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.pd_schedule_primary["Observers"].pagerduty_schedule.schedule will be updated in-place
  ~ resource "pagerduty_schedule" "schedule" {
        id          = "PEFC91G"
        name        = "Observers_primary"
        # (2 unchanged attributes hidden)

      ~ layer {
            id                           = "PZW4R4J"
          ~ name                         = "Night on-call" -> "Day on-call"
          ~ users                        = [
                # (1 unchanged element hidden)
                "PY3OLWJ",
              + "P0YY7L2",
                "PCG7S2F",
                # (2 unchanged elements hidden)
            ]
            # (3 unchanged attributes hidden)

          ~ restriction {
              ~ start_day_of_week = 3 -> 1
                # (3 unchanged attributes hidden)
            }
        }
      ~ layer {
            id                           = "PVJGQSW"
          ~ name                         = "Day on-call" -> "Night on-call"
          ~ rotation_turn_length_seconds = 86400 -> 604800
          ~ users                        = [
              - "PW2V0SI",
                "PY3OLWJ",
              + "P0YY7L2",
                "PCG7S2F",
                # (1 unchanged element hidden)
                "PNA5ZUT",
              + "PW2V0SI",
            ]
            # (2 unchanged attributes hidden)

          ~ restriction {
              ~ start_day_of_week = 1 -> 3
                # (3 unchanged attributes hidden)
            }
        }
        # (1 unchanged block hidden)
    }

  # module.pd_schedule_secondary["Observers"].pagerduty_schedule.schedule will be updated in-place
  ~ resource "pagerduty_schedule" "schedule" {
        id          = "PB6R8PT"
        name        = "Observers_secondary"
        # (2 unchanged attributes hidden)

      ~ layer {
            id                           = "P81JOOT"
          ~ name                         = "Night on-call" -> "Day on-call"
          ~ users                        = [
              + "PCG7S2F",
              + "PGDC19E",
                "PNA5ZUT",
                # (1 unchanged element hidden)
                "PY3OLWJ",
              - "PCG7S2F",
              - "PGDC19E",
              + "P0YY7L2",
            ]
            # (3 unchanged attributes hidden)

          ~ restriction {
              ~ start_day_of_week = 3 -> 1
                # (3 unchanged attributes hidden)
            }
        }
      ~ layer {
            id                           = "P0GN9JD"
          ~ name                         = "Day on-call" -> "Night on-call"
          ~ rotation_turn_length_seconds = 86400 -> 604800
          ~ users                        = [
              + "PGDC19E",
                "PNA5ZUT",
                # (1 unchanged element hidden)
                "PY3OLWJ",
              + "P0YY7L2",
                "PCG7S2F",
              - "PGDC19E",
            ]
            # (2 unchanged attributes hidden)

          ~ restriction {
              ~ start_day_of_week = 1 -> 3
                # (3 unchanged attributes hidden)
            }
        }
        # (1 unchanged block hidden)
    }

Plan: 0 to add, 2 to change, 0 to destroy.

Steps to Reproduce

Unfortunately, due to its nature, the issue is sporadic. Some runs result in a change, while others do not. Generally speaking however, running terraform plan multiple times results in some of the runs having changes.

References

Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:

• I think that perhaps #249 and #200 could be related

[AstritCepele]

We also have been experiencing the same issue for a while now. Especially when we add or remove a user the sorting logic is incorrect.

[NargiT]

In my case it does not detect anything if I add ignore on users list.

  lifecycle {
    ignore_changes = [
      layer[0].users,
      layer[1].users,
      layer[2].users
    ]
  }

looks like sub groups are not managed correctly as everything is ignored.

https://stackoverflow.com/questions/48243968/terraform-ignore-changes-and-sub-blocks

[austinpray-mixpanel]

Seeing same thing here. Removing 2 users from a schedule:

Ended up with the schedule all jumbled with the R suffix id in the screenshot being second to last rather than last.