azurerm_cdn_frontdoor_rule_set destroyed before azurerm_cdn_frontdoor_route is updated

[gitfrs]

Is there an existing issue for this?

• [X] I have searched the existing issues

Community Note

• Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

1.3.9

AzureRM Provider Version

3.45

Affected Resource(s)/Data Source(s)

azurerm_cdn_frontdoor_rule_set, azurerm_cdn_frontdoor_route

Terraform Configuration Files

locals {
  routes = flatten([
    for cd in var.endpoint_settings : [
      for r in cd.routes : [
        {
          prefix     = cd.prefix,
          dns_zone   = cd.dns_zone,
          route_name = r.route_name,
          route      = r
          rule_sets  = concat(flatten(r.route_settings.rule_set_to_apply.creating_rule_sets_refs), flatten(r.route_settings.rule_set_to_apply.existing_rule_sets)) 
        }
      ] 
    ] 
  ])
  rules_per_rule_set = flatten([
    for rule_set in var.rule_sets : [
      for rule in rule_set.rules : merge({
        rule_set_name = rule_set.name
      }, rule)
    ]
  ])
}

resource "azurerm_cdn_frontdoor_route" "those" {
  count                         = length(local.routes)
  name                          = "${var.environment}-${local.routes[count.index].prefix}-${local.routes[count.index].route_name}-${count.index}"
  cdn_frontdoor_endpoint_id     = azurerm_cdn_frontdoor_endpoint.those[index(azurerm_cdn_frontdoor_endpoint.those.*.name, "${var.environment}-${local.routes[count.index].prefix}-default-route")].id
  cdn_frontdoor_origin_group_id = azurerm_cdn_frontdoor_origin_group.those[index(azurerm_cdn_frontdoor_origin_group.those.*.name, "${var.environment}-${local.routes[count.index].prefix}-${local.routes[count.index].route_name}")].id
  cdn_frontdoor_origin_ids      = [for i in azurerm_cdn_frontdoor_origin.those : i.id if i.cdn_frontdoor_origin_group_id == azurerm_cdn_frontdoor_origin_group.those[index(azurerm_cdn_frontdoor_origin_group.those.*.name, "${var.environment}-${local.routes[count.index].prefix}-${local.routes[count.index].route_name}")].id]
  cdn_frontdoor_rule_set_ids = local.routes[count.index].rule_sets != null ? [for rs in local.routes[count.index].rule_sets : azurerm_cdn_frontdoor_rule_set.those[rs].id] : []
  ...
}

resource "azurerm_cdn_frontdoor_rule_set" "those" {
  for_each = try({ for rule_set in var.rule_sets : rule_set.name => rule_set }, {})
  name                     = each.value.name
  cdn_frontdoor_profile_id = var.cdn_frontdoor_profile_ref != null ? var.cdn_frontdoor_profile_ref.id : azurerm_cdn_frontdoor_profile.this[0].id
}

Debug Output/Panic Output

######################
# PLAN
######################
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place
  - destroy


Terraform will perform the following actions:

  # module.cdn_frontdoor.azurerm_cdn_frontdoor_route.those[0] will be updated in-place
  ~ resource "azurerm_cdn_frontdoor_route" "those" {
      ~ cdn_frontdoor_rule_set_ids      = [
          - "/.../ruleSets/rsheader",
          - "/.../ruleSets/rsredirect",
          - "/.../ruleSets/ruleset1",
          - "/.../ruleSets/ruleset2",
        ]
        id                              = "/..."
        name                            = "redacted"
        # (10 unchanged attributes hidden)
    }

  # module.cdn_frontdoor.azurerm_cdn_frontdoor_rule_set.those["ruleset1"] will be destroyed
  # (because key ["ruleset1"] is not in for_each map)
  - resource "azurerm_cdn_frontdoor_rule_set" "those" {
      - cdn_frontdoor_profile_id = "redacted" -> null
      - id                       = "redacted" -> null
      - name                     = "ruleset1" -> null
    }

  # module.cdn_frontdoor.azurerm_cdn_frontdoor_rule_set.those["ruleset2"] will be destroyed
  # (because key ["ruleset2"] is not in for_each map)
  - resource "azurerm_cdn_frontdoor_rule_set" "those" {
      - cdn_frontdoor_profile_id = "redacted" -> null
      - id                       = "redacted" -> null
      - name                     = "ruleset2" -> null
    }

######################
# APPLY
######################
module.cdn_frontdoor.azurerm_cdn_frontdoor_rule_set.those["ruleset1"]: Destroying... [id=/.../ruleSets/ruleset1]
╷
│ Error: deleting Front Door Rule Set: (Rule Set Name "ruleset1" / Profile Name "..." / Resource Group "..."): cdn.RuleSetsClient
# Delete: Failure sending request: StatusCode=400 -- Original Error: Code="BadRequest" Message="This resource is still associated with a route.
# Please delete the association with the route first before deleting this resource."

│ Error: deleting Front Door Rule Set: (Rule Set Name "ruleset2" / Profile Name "..." / Resource Group "..."): cdn.RuleSetsClient
# Delete: Failure sending request: StatusCode=400 -- Original Error: Code="BadRequest" Message="This resource is still associated with a route.
# Please delete the association with the route first before deleting this resource."

##[error]Terraform command 'apply' failed with exit code '1'.
##[error]╷
│ Error: deleting Front Door Rule Set: (Rule Set Name "ruleset1" / Profile Name "..." / Resource Group "..."): cdn.RuleSetsClient
# Delete: Failure sending request: StatusCode=400 -- Original Error: Code="BadRequest" Message="This resource is still associated with a route.
# Please delete the association with the route first before deleting this resource."

│ Error: deleting Front Door Rule Set: (Rule Set Name "ruleset2" / Profile Name "..." / Resource Group "..."): cdn.RuleSetsClient
# Delete: Failure sending request: StatusCode=400 -- Original Error: Code="BadRequest" Message="This resource is still associated with a route.
# Please delete the association with the route first before deleting this resource."

Finishing: terraform apply

Expected Behaviour

The route should be updated before trying to destroy the rule sets.

Actual Behaviour

There's an attempt at destroying the rule sets first that fails because the rul set is still associated with a route.

Steps to Reproduce

Deleting rule sets that are already associated with a route

Important Factoids

No response

References

No response

[gitfrs]

Hey, I made some further attempts and it seems that the azurerm_cdn_frontdoor_route cannot unassign rules sets. I tried just removing assigned rules sets from a route. It ran successfully but nothing has changed on the route itself:

module.cdn_frontdoor.azurerm_cdn_frontdoor_route.those[0]: Modifying... [id=/==redacted==]
module.cdn_frontdoor.azurerm_cdn_frontdoor_route.those[0]: Still modifying... [id=/==redacted==, 10s elapsed]
module.cdn_frontdoor.azurerm_cdn_frontdoor_route.those[0]: Modifications complete after 11s [id=/==redacted==]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
Finishing: terraform apply

[WodansSon]

@gitfrs, I believe this is due to the module using strings instead of hard references to the actual resources. That said, I will look into this issue. Thanks for opening the issue. 🚀

[gitfrs]

@WodansSon Hello Thanks for looking into it. I made some more testing and tried using only references:

existing_rule_sets = [data.azurerm_cdn_frontdoor_rule_set.rsheader.id, data.azurerm_cdn_frontdoor_rule_set.rsredirect.id]

locals { ...
  rule_sets  = r.route_settings.rule_set_to_apply.existing_rule_sets 
}

resource "azurerm_cdn_frontdoor_route" "those" { ...
  cdn_frontdoor_rule_set_ids    = local.routes[count.index].rule_sets
}

The result is the same is above:

module.cdn_frontdoor.azurerm_cdn_frontdoor_route.those[0]: Modifying... [id=/==redacted==]
module.cdn_frontdoor.azurerm_cdn_frontdoor_route.those[0]: Still modifying... [id=/==redacted==, 10s elapsed]
module.cdn_frontdoor.azurerm_cdn_frontdoor_route.those[0]: Modifications complete after 11s [id=/==redacted==]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
Finishing: terraform apply

[WodansSon]

@gitfrs, I don't think that will work because, ultimately, the ID's are still getting abstracted to string values. In my repro for this issue I have a hard reference to the actual resource (e.g. cdn_frontdoor_rule_set_ids = [azurerm_cdn_frontdoor_rule_set.test.id]) and I do not see the errors that you are seeing. I will try some different configurations and see if I can get a repro of this issue.

resource "azurerm_cdn_frontdoor_route" "contoso" {
  name                          = "route-contoso"
  cdn_frontdoor_endpoint_id     = azurerm_cdn_frontdoor_endpoint.contoso.id
  cdn_frontdoor_origin_group_id = azurerm_cdn_frontdoor_origin_group.test.id
  cdn_frontdoor_origin_ids      = [azurerm_cdn_frontdoor_origin.test.id]
  enabled                       = true

  https_redirect_enabled     = true
  forwarding_protocol        = "HttpsOnly"
  patterns_to_match          = ["/contoso"]
  supported_protocols        = ["Http", "Https"]
  cdn_frontdoor_rule_set_ids = [azurerm_cdn_frontdoor_rule_set.test.id]

  cdn_frontdoor_custom_domain_ids = [azurerm_cdn_frontdoor_custom_domain.contoso.id]
  link_to_default_domain          = false

  cache {
    compression_enabled           = true
    content_types_to_compress     = ["text/html", "text/javascript", "text/xml"]
    query_strings                 = ["account", "settings", "foo", "bar"]
    query_string_caching_behavior = "IgnoreSpecifiedQueryStrings"
  }
}

[WodansSon]

@gitfrs, I just made a configuration that exhibits the behavior you mention in your issue. It looks like I am going to have to expose a new association resource between the azurerm_cdn_frontdoor_route and the azurerm_cdn_frontdoor_rule_set resources to fix this issue unfortunately. The way this would work is when the association resource is being deleted, since it will have a reference to both the route and rule set, it will update the route removing the rule set from it's configuration and then delete the rule set to avoid the error when deleting the route which would have been still referencing the rule set. This can get tricky, so I will have to put a lot of thought into how to do this correctly...

[gitfrs]

Hello @WodansSon . Thanks for looking into it.

I will follow up and in the meantime, will try to use hard references as you suggested.

Thanks again !

[WodansSon]

@gitfrs, I don't think the hard reference will help here TBH, because the reference will still exist. It worked in my case because I did a destroy instead of just deleting of a resource. I honestly think the only way to fix this is to inject a new association resource into the mix, which sucks, but I think that might be the only way around this issue. Let me know how it turns out as that will give me an additional data point to help me ascertain what is actually happening here. 🚀

[ekasprzak]

Hello, just checked it on 3.62.1 and the result is the same so is there any info/timeline on when the PR that might fix it would be merged?

[WodansSon]

@ekasprzak, sorry no I cannot provide an ETA on having the fix merged. @jackofallops for awareness.

[Planche95]

Hi, any updates on that one?

[SebastianBalle]

The problem remains in 3.98.0

[RolfMoleman]

@reganlives willl need to bare this in mind

[m1rl0k]

can someone please tell me how to add security headers to the route with this