terraform-provider-cloudflare icon indicating copy to clipboard operation
terraform-provider-cloudflare copied to clipboard

cloudflare_api_token resource always shows changes (drift)

Open boillodmanuel opened this issue 7 months ago • 5 comments

Confirmation

  • [x] This is a bug with an existing resource and is not a feature request or enhancement. Feature requests should be submitted with Cloudflare Support or your account team.
  • [x] I have searched the issue tracker and my issue isn't already found.
  • [x] I have replicated my issue using the latest version of the provider and it is still present.

Terraform and Cloudflare provider version

Terraform v1.9.8
on darwin_arm64
+ provider registry.terraform.io/cloudflare/cloudflare v5.3.0

Affected resource(s)

  • cloudflare_api_token

Terraform configuration files

terraform {
  required_version = ">= 1.0"
  required_providers {
    cloudflare = {
      source  = "cloudflare/cloudflare"
      version = "~>5.3"
    }
  }
  backend "local" {}
}

provider "cloudflare" {
  email   = local.CLOUDFLARE_EMAIL
  api_key = local.CLOUDFLARE_API_KEY
}


data "cloudflare_api_token_permission_groups_list" "all" {
}

locals {
  api_token_zone_permissions_groups_map = {
    for perm in data.cloudflare_api_token_permission_groups_list.all.result :
    perm.name => perm.id
    if contains(perm.scopes, "com.cloudflare.api.account.zone")
  }
}

resource "cloudflare_api_token" "test_api_token" {
  name   = "test-cf-v5-api-token"
  status = "active"

  policies = [{
    effect = "allow"
    permission_groups = [
      { "id" = local.api_token_zone_permissions_groups_map["DNS Write"] },
      { "id" = local.api_token_zone_permissions_groups_map["Zone Read"] },
    ]
    resources = {
      "com.cloudflare.api.account.${local.ACCOUNT_ID}" = "*"
    }
  }]
}

Link to debug output

no

Panic output

No response

Expected output

No changes detected

Actual output

Terraform will perform the following actions:

  # cloudflare_api_token.test_api_token will be updated in-place
  ~ resource "cloudflare_api_token" "test_api_token" {
      + condition    = (known after apply)
        id           = "de493ea5bbd25d3b48f6f66abe35fdde"
      ~ issued_on    = "2025-05-05T09:22:28Z" -> (known after apply)
      + last_used_on = (known after apply)
      ~ modified_on  = "2025-05-05T09:37:52Z" -> (known after apply)
        name         = "test-cf-v5-api-token"
      ~ policies     = [
          ~ {
              ~ id                = "2d6f78b5a2f041a9967083e295e01c87" -> (known after apply)
              ~ permission_groups = [
                  ~ {
                      ~ id   = "c8fed203ed3043cba015a93ad1616f1f" -> "4755a26eedb94da69e1066d98aa820be"
                      ~ name = "Zone Read" -> (known after apply)
                    },
                  ~ {
                      ~ id   = "4755a26eedb94da69e1066d98aa820be" -> "c8fed203ed3043cba015a93ad1616f1f"
                      ~ name = "DNS Write" -> (known after apply)
                    },
                ]
                # (2 unchanged attributes hidden)
            },
        ]
      ~ value        = (sensitive value)
        # (1 unchanged attribute hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Steps to reproduce

terraform apply with above code

Additional factoids

No response

References

No response

boillodmanuel avatar May 05 '25 09:05 boillodmanuel

We have assigned this to a team internally to take a look. Thank you!

jhutchings1 avatar May 12 '25 21:05 jhutchings1

it looks like it want's to be sorted by permission id, that is rather inconvenient

zuozp8 avatar May 20 '25 13:05 zuozp8

Also having the issue in https://github.com/jenkins-infra/helpdesk/issues/4601. For the time being, we've set add to sort manually the list in our code until it is fixed:

Reproduction case (e.g. drifting):


terraform {
  required_version = ">= 1.12, <1.13"
  required_providers {
    cloudflare = {
      source = "cloudflare/cloudflare"
    }
  }
}

provider "cloudflare" {
}

data "cloudflare_api_token_permission_groups_list" "all" {
}

locals {
  # https://developers.cloudflare.com/fundamentals/api/reference/permissions/#zone-permissions
  api_token_zone_permissions_groups_map = {
    for perm in data.cloudflare_api_token_permission_groups_list.all.result :
    perm.name => perm.id
    if contains(perm.scopes, "com.cloudflare.api.account.zone")
  }
}

variable "account_id" {
  type = string
}

resource "cloudflare_api_token" "terraform" {
  name = "terraform"

  policies = [{
    effect = "allow"
    resources = {
      "com.cloudflare.api.account.${var.account_id}" = "*"
    }
    permission_groups = [
      { id = local.api_token_zone_permissions_groups_map["Zone Settings Write"] },
      { id = local.api_token_zone_permissions_groups_map["Zone Write"] },
      { id = local.api_token_zone_permissions_groups_map["DNS Write"] },
      { id = local.api_token_zone_permissions_groups_map["SSL and Certificates Write"] },
      { id = local.api_token_zone_permissions_groups_map["Logs Write"] },
    ]
  }]

  condition = {
    request_ip = {
      in = ["127.0.0.1/32"]
    }
  }

  expires_on = "2025-06-17T00:00:00Z"
}

Manual fix (not drifting, but hard to maintain on long term, unable to templatize/factorize/modularize):


terraform {
  required_version = ">= 1.12, <1.13"
  required_providers {
    cloudflare = {
      source = "cloudflare/cloudflare"
    }
  }
}

provider "cloudflare" {
}

data "cloudflare_api_token_permission_groups_list" "all" {
}

locals {
  # https://developers.cloudflare.com/fundamentals/api/reference/permissions/#zone-permissions
  api_token_zone_permissions_groups_map = {
    for perm in data.cloudflare_api_token_permission_groups_list.all.result :
    perm.name => perm.id
    if contains(perm.scopes, "com.cloudflare.api.account.zone")
  }
}

variable "account_id" {
  type = string
}

resource "cloudflare_api_token" "terraform" {
  name = "terraform"

  policies = [{
    effect = "allow"
    resources = {
      "com.cloudflare.api.account.${var.account_id}" = "*"
    }
    permission_groups = [
        # Order matter until https://github.com/cloudflare/terraform-provider-cloudflare/issues/5548 is fixed
        { id = local.api_token_zone_permissions_groups_map["Zone Settings Write"] },
        { id = local.api_token_zone_permissions_groups_map["Zone Write"] },
        { id = local.api_token_zone_permissions_groups_map["SSL and Certificates Write"] },
        { id = local.api_token_zone_permissions_groups_map["Logs Write"] },
        { id = local.api_token_zone_permissions_groups_map["DNS Write"] },
    ]
  }]

  condition = {
    request_ip = {
      in = ["127.0.0.1/32"]
    }
  }

  expires_on = "2025-06-17T00:00:00Z"
}

dduportal avatar Jun 02 '25 11:06 dduportal

Just a quick message to highlight this issue that is not fixed in today 5.6.0 version. Any updates @jhutchings1 @jacobbednarz ?

boillodmanuel avatar Jun 17 '25 08:06 boillodmanuel

There's been no action on this one yet. The team is expecting to start work on the IAM related Terraform tickets starting in July.

jhutchings1 avatar Jun 19 '25 01:06 jhutchings1

Hey all, the order of permission groups on an api_token policy should now not affect the plan. Please try again with the latest 5.7.0 provider release.

steve-thousand avatar Jul 15 '25 19:07 steve-thousand

@jhutchings1 This issue is still present for us when using version 5.7.1 of the Terraform provider - we had to sort permission group IDs as a workaround. Can this issue be reopened?

nslusher-sf avatar Jul 30 '25 15:07 nslusher-sf

@steve-thousand can you have a look at @nslusher-sf 's comment above? Thanks!

jhutchings1 avatar Jul 30 '25 18:07 jhutchings1

Can confirm that this is still an issue in version 5.8.2 of the Terraform provider.

TyranVdMerwePaytec avatar Aug 05 '25 18:08 TyranVdMerwePaytec

We believe we have fixed this and it should be coming soon

steve-thousand avatar Aug 12 '25 22:08 steve-thousand

We believe in the latest 5.8.4 we have now fixed this issue of multiple permission groups on the same policy. Can you upgrade to the latest provider version and try again?

steve-thousand avatar Aug 16 '25 20:08 steve-thousand

@steve-thousand using 5.8.4 I am getting an issue with re-creating an api token. here is a reproduction

data "cloudflare_api_token_permission_groups_list" "all" {
}

locals {
  api_token_r2_permissions_groups_map = {
    for perm in data.cloudflare_api_token_permission_groups_list.all.result :
    perm.name => perm.id
    if contains(perm.scopes, "com.cloudflare.edge.r2.bucket")
  }
}

resource "cloudflare_api_token" "r2_media_token" {
  name = "tertour-media-bucket-rw-token"
  // Use object read/write permissions for the specific bucket
  policies = [{
    effect = "allow"
    resources = {
      "com.cloudflare.edge.r2.bucket.${local.account_id}_${local.media_bucket_jurisdiction}_${cloudflare_r2_bucket.media_bucket.name}" = "*"
    }
    # permissions groups can be found here: https://api.cloudflare.com/client/v4/user/tokens/permission_groups
    permission_groups = [
      { id = local.api_token_r2_permissions_groups_map["Workers R2 Storage Bucket Item Write"] }
    ]
  }]
}

I am running terraform with a -replace flag for the token. terrafrom deleted the token but was unable to re-create it.

I get the following error:

POST "https://api.cloudflare.com/client/v4/user/tokens": 400 Bad Request
│ {"success":false,"errors":[{"code":400,"message":"- These rules must pass for
│ `{ \"policies\": { { \"permission_groups\": ..., \"resources\": ... } } }`\n
│ - name must be present"}],"messages":[],"result":null}

In the debug output i can see that the permission groups are empty and not sent to the api

{"policies":[{"permission_groups":[{}],"resources":{"com.cloudflare.edge.r2.bucket.[REDACTED]_eu_tertour-media":null}}]}

Also assigning a name directly violates the resource schema as name is a read-only property

meaning this would throw an error

resource "cloudflare_api_token" "r2_media_token" {
  name = "tertour-media-bucket-rw-token"
  
  policies = [{
    effect = "allow"
    permission_groups = [{
      id = "2efd5506f9c8494dacb1fa10a3e7d5b6"
      name = "Workers R2 Storage Bucket Item Write"
    }]
    resources = {
      "com.cloudflare.edge.r2.bucket.${local.account_id}_${local.media_bucket_jurisdiction}_${cloudflare_r2_bucket.media_bucket.name}" = "*"
    }
  }]
}

Any idea how to re-mediate to this?

khawarizmus avatar Aug 19 '25 15:08 khawarizmus

@steve-thousand using 5.8.4 I am getting an issue with re-creating an api token. here is a reproduction

data "cloudflare_api_token_permission_groups_list" "all" { }

locals { api_token_r2_permissions_groups_map = { for perm in data.cloudflare_api_token_permission_groups_list.all.result : perm.name => perm.id if contains(perm.scopes, "com.cloudflare.edge.r2.bucket") } }

resource "cloudflare_api_token" "r2_media_token" { name = "tertour-media-bucket-rw-token" // Use object read/write permissions for the specific bucket policies = [{ effect = "allow" resources = { "com.cloudflare.edge.r2.bucket.${local.account_id}${local.media_bucket_jurisdiction}${cloudflare_r2_bucket.media_bucket.name}" = "*" } # permissions groups can be found here: https://api.cloudflare.com/client/v4/user/tokens/permission_groups permission_groups = [ { id = local.api_token_r2_permissions_groups_map["Workers R2 Storage Bucket Item Write"] } ] }] } I am running terraform with a -replace flag for the token. terrafrom deleted the token but was unable to re-create it.

I get the following error:

POST "https://api.cloudflare.com/client/v4/user/tokens": 400 Bad Request
│ {"success":false,"errors":[{"code":400,"message":"- These rules must pass for
│ `{ \"policies\": { { \"permission_groups\": ..., \"resources\": ... } } }`\n
│ - name must be present"}],"messages":[],"result":null}

In the debug output i can see that the permission groups are empty and not sent to the api

{"policies":[{"permission_groups":[{}],"resources":{"com.cloudflare.edge.r2.bucket.[REDACTED]_eu_tertour-media":null}}]}

Also assigning a name directly violates the resource schema as name is a read-only property

meaning this would throw an error

resource "cloudflare_api_token" "r2_media_token" { name = "tertour-media-bucket-rw-token"

policies = [{ effect = "allow" permission_groups = [{ id = "2efd5506f9c8494dacb1fa10a3e7d5b6" name = "Workers R2 Storage Bucket Item Write" }] resources = { "com.cloudflare.edge.r2.bucket.${local.account_id}${local.media_bucket_jurisdiction}${cloudflare_r2_bucket.media_bucket.name}" = "*" } }] } Any idea how to re-mediate to this?

downgrading to 5.5.0 fixes my issue

khawarizmus avatar Aug 20 '25 02:08 khawarizmus

@khawarizmus I am unable to recreate either of these issues.

If I am understanding you correctly you are reporting two different issues:

  1. Attempting to recreate an api-token using the terraform apply -replace cloudflare_api_token.r2_media_token results in a 400 on recreate due to empty permission groups (thank you for providing the debug output)
  2. The name field on an api-token is reportedly readonly

Regarding 1) Are you recreating an api-token that was originally created in version 5.8.4?

Regarding 2) the name field of an api token should not be readonly. Are you getting an error output by the provider when trying to set it?

Could you maybe provide the cloudflare provider block from your .terraform.lock.hcl?

steve-thousand avatar Aug 20 '25 17:08 steve-thousand