hashicorp
google_compute_security_policy rules are always recreated
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
- Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
- If you are interested in working on this issue or have submitted a pull request, please leave a comment.
- If an issue is assigned to the
modular-magicianuser, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned tohashibot, a community member has claimed the issue already.
Terraform Version
❯ terraform -v
Terraform v1.5.4
on darwin_arm64
[...]
+ provider registry.terraform.io/hashicorp/google v5.10.0
+ provider registry.terraform.io/hashicorp/google-beta v5.10.0
[...]
The same versions were also tested with amd64
Affected Resource(s)
- google_compute_security_policy
Using resource through module GoogleCloudPlatform/cloud-armor/google version 2.0.1
Expected Behavior
Terraform plan should be idempotent after terraform apply
Actual Behavior
Terraform always wants to recreate all rules in the security policy even if no changes are made and after applying and checking that all changes have been made.
Example of one rule:
# module.<mymodule>.module.security_policy[0].google_compute_security_policy.policy will be updated in-place
~ resource "google_compute_security_policy" "policy" {
id = "projects/<project-id>/global/securityPolicies/<security-policy-id>"
name = "<security-policy-name>"
# (5 unchanged attributes hidden)
- rule {
- action = "throttle" -> null
- description = "Throttle per api rule for <redacted>" -> null
- preview = false -> null
- priority = 80030 -> null
- match {
- expr {
- expression = <<-EOT
has(request.headers['Host'])
&& (
request.headers['Host'].contains('<redacted>')
) &&
request.path.matches('^/.*$')
EOT -> null
}
}
[...]
+ rule {
+ action = "throttle"
+ description = "Throttle per api rule for <redacted>"
+ preview = false
+ priority = 80030
+ match {
+ expr {
+ expression = <<-EOT
has(request.headers['Host'])
&& (
request.headers['Host'].contains('<redacted>')
) &&
request.path.matches('^/.*$')
EOT
}
}
+ rate_limit_options {
+ conform_action = "allow"
+ enforce_on_key = "XFF_IP"
+ exceed_action = "deny(429)"
+ rate_limit_threshold {
+ count = 500
+ interval_sec = 60
}
}
}
References
https://github.com/hashicorp/terraform-provider-google/issues/9084 The same issue as described and related issues
b/318850546
Facing similar issue with preconfigured_waf_config block being present in new rules being added via the GCP console.
Seems like its happening only with newly created rules. I have a security policy which was created via the GCP console & has a total of 22 rules(again created via the GCP console), one of these rules was added relatively recently when compared to remaining 21.
I am working on terraform import to construct the state of this security policy and I see that after import for this one particular rule, terraform always tries to delete the preconfigured_waf_config block & as a result re-creates that rule. All the other 21 are just fine.
If I run terraform apply and then immediately run terraform plan I see that it again tries to delete the preconfigured_waf_config block.
To confirm my suspicion that this is been happening only with relatively new rules, I performed the below on the mentioned order:
- Created a new security policy and added 3 rules to it via the GCP console.
- Imported the state using
terraform import. - Ran
terraform plan(this showed that thepreconfigured_waf_configblock in the existing rule has to be deleted & hence the rule will be deleted and recreated.) - Ran
terraform apply(It was successful) - Immediately ran
terraform planagain & had same observations as in step 3.
$ terraform plan -out test -var-file ./vars/dev.tfvars
module.cloud-armor["test"].google_compute_security_policy.policy: Refreshing state... [id=projects/<project_id>/global/securityPolicies/test]
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
# module.cloud-armor["test"].google_compute_security_policy.policy will be updated in-place
~ resource "google_compute_security_policy" "policy" {
id = "projects/<project_id>/global/securityPolicies/test"
name = "test"
# (5 unchanged attributes hidden)
- adaptive_protection_config {
- layer_7_ddos_defense_config {
- enable = false -> null
}
}
- rule {
- action = "allow" -> null
- description = "test1" -> null
- preview = false -> null
- priority = 10 -> null
- match {
- versioned_expr = "SRC_IPS_V1" -> null
- config {
- src_ip_ranges = [
- "1.1.1.1",
- "2.2.2.2",
] -> null
}
}
- preconfigured_waf_config {
}
}
- rule {
- action = "allow" -> null
- description = "test2" -> null
- preview = false -> null
- priority = 20 -> null
- match {
- versioned_expr = "SRC_IPS_V1" -> null
- config {
- src_ip_ranges = [
- "3.3.3.3",
- "4.4.4.4",
] -> null
}
}
- preconfigured_waf_config {
}
}
- rule {
- action = "deny" -> null
- description = "Default rule, higher priority overrides it" -> null
- preview = false -> null
- priority = 2147483647 -> null
- match {
- versioned_expr = "SRC_IPS_V1" -> null
- config {
- src_ip_ranges = [
- "*",
] -> null
}
}
- preconfigured_waf_config {
}
}
- rule {
- action = "deny(502)" -> null
- description = "test3" -> null
- preview = false -> null
- priority = 30 -> null
- match {
- versioned_expr = "SRC_IPS_V1" -> null
- config {
- src_ip_ranges = [
- "5.5.5.5",
- "6.6.6.6",
] -> null
}
}
- preconfigured_waf_config {
}
}
+ rule {
+ action = "allow"
+ description = "test1"
+ preview = false
+ priority = 10
+ match {
+ versioned_expr = "SRC_IPS_V1"
+ config {
+ src_ip_ranges = [
+ "1.1.1.1",
+ "2.2.2.2",
]
}
}
}
+ rule {
+ action = "allow"
+ description = "test2"
+ preview = false
+ priority = 20
+ match {
+ versioned_expr = "SRC_IPS_V1"
+ config {
+ src_ip_ranges = [
+ "3.3.3.3",
+ "4.4.4.4",
]
}
}
}
+ rule {
+ action = "deny"
+ description = "Default rule, higher priority overrides it"
+ preview = (known after apply)
+ priority = 2147483647
+ match {
+ versioned_expr = "SRC_IPS_V1"
+ config {
+ src_ip_ranges = [
+ "*",
]
}
}
}
+ rule {
+ action = "deny(502)"
+ description = "test3"
+ preview = false
+ priority = 30
+ match {
+ versioned_expr = "SRC_IPS_V1"
+ config {
+ src_ip_ranges = [
+ "5.5.5.5",
+ "6.6.6.6",
]
}
}
}
# (1 unchanged block hidden)
}
Tried a couple of more times and repeatedly saw this issue occurring.
When I create the policy & rules completely using terraform then I dont see this happening(Using module version v1.1.0).
There might be some slight overlap here https://github.com/hashicorp/terraform-provider-google/issues/17288
To prevent my rules from being recreated each time, I had to specify these values in each rule that contained a match.expr (I just have two rules):
preview = false
preconfigured_waf_config {}
I also have a default rule with a match.versioned_expr that didn't need these set, as terraform doesn't think that rule needs to be replaced.)
Here are my versions:
$ terraform -version
Terraform v1.5.7
on darwin_amd64
+ provider registry.terraform.io/hashicorp/google v5.14.0
+ provider registry.terraform.io/hashicorp/google-beta v5.14.0
+ provider registry.terraform.io/hashicorp/random v3.6.0
(I am using the google-beta provider for my google_compute_security_policy.)
This bug makes managing cloud armor rules in terraform unusable, ie dangerous in production
To prevent my rules from being recreated each time, I had to specify these values in each rule that contained a
match.expr(I just have two rules):preview = false preconfigured_waf_config {}
This didn't make any difference for me, I still see the rules being recreated.
~~I had a single rule that was constantly changing with preconfigured_waf_config. I changed the priority of that rule in the UI manually, then reapplied the TF. No more reoccurring changes after that.~~
Edit: that didn't last long. Came back within a couple days.
Facing the same problem.
# module.security_policy.google_compute_security_policy.policy will be updated in-place
~ resource "google_compute_security_policy" "policy" {
id = "projects/divdot-production/global/securityPolicies/frontend-host-policy"
name = "frontend-host-policy"
# (4 unchanged attributes hidden)
- rule {
- action = "allow" -> null
- description = "Allow Requests only from firebase frontend" -> null
- preview = false -> null
- priority = 1000 -> null
- match {
- expr {
- expression = <<-EOT
request.headers['host'].lower().contains('example.com')
EOT -> null
}
}
}
+ rule {
+ action = "allow"
+ description = "Allow Requests only from firebase frontend"
+ preview = false
+ priority = 1000
+ match {
+ expr {
+ expression = <<-EOT
request.headers['host'].lower().contains('example.com')
EOT
}
}
}
# (3 unchanged blocks hidden)
}
These rules were initially created using terraform and rerunning terraform apply recreates them every time.
Hi guys,
I'm also facing the same issue, Any news regarding this?
For me it seems that the cause is in the preconfigured_waf_config for the rules and in layer_7_ddos_defense_config
~ resource "google_compute_security_policy" "policy" {
id = "projects/agp-exchange-prod-0u/global/securityPolicies/grafana-office"
name = "grafana-office"
# (5 unchanged attributes hidden)
- adaptive_protection_config {
- layer_7_ddos_defense_config {
- enable = false -> null
# (1 unchanged attribute hidden)
}
}
- rule {
- action = "allow" -> null
- description = "Allow only office CIDR" -> null
- preview = false -> null
- priority = 0 -> null
- match {
- versioned_expr = "SRC_IPS_V1" -> null
- config {
- src_ip_ranges = [
- "x.x.x.x/29",
] -> null
}
}
- preconfigured_waf_config {
}
}
- rule {
- action = "deny(403)" -> null
- description = "Default rule, higher priority overrides it" -> null
- preview = false -> null
- priority = 2147483647 -> null
- match {
- versioned_expr = "SRC_IPS_V1" -> null
- config {
- src_ip_ranges = [
- "*",
] -> null
}
}
- preconfigured_waf_config {
}
}
+ rule {
+ action = "allow"
+ description = "Allow only office CIDR"
+ preview = false
+ priority = 0
+ match {
+ versioned_expr = "SRC_IPS_V1"
+ config {
+ src_ip_ranges = [
+ "x.x.x.x/29",
]
}
}
}
+ rule {
+ action = "deny(403)"
+ description = "Default rule, higher priority overrides it"
+ preview = (known after apply)
+ priority = 2147483647
+ match {
+ versioned_expr = "SRC_IPS_V1"
+ config {
+ src_ip_ranges = [
+ "*",
]
}
}
}
# (1 unchanged block hidden)
}
This issue is blocking https://github.com/GoogleCloudPlatform/terraform-google-cloud-armor/issues/94
also experience the exact same issue ➕ 😢
terraform_version: 1.8.5
..
- Installed hashicorp/google v5.38.0 (signed by HashiCorp)
- Installing hashicorp/google-beta v5.38.0...
- Installed hashicorp/google-beta v5.38.0 (signed by HashiCorp)
I have the same issue, I believe this is because the API to fetch the security policy returns the rules in an unsorted order, so I believe this method should fix it
Suggested workaround while this is broken:
Add a lifecycle ignore changes block to google_compute_security_policy:
lifecycle { ignore_changes = [rule] }
Define your rules using the google_compute_security_policy_rule resource.
Your rules will be defined by a different resource, and you will ignore the policy's desire to delete them
