`vault_gcp_auth_backend_role` constantly shows changes to new `token_policies` attribute

[zx8]

Overview

After upgrading this provider from v2.3.0 to v2.5.0 and updating my vault_gcp_auth_backend_role resource to use token_policies instead of policies, constantly see changes to token_policies, with the policies attribute still being displayed (unchanged) in the output:

Resource

resource "vault_gcp_auth_backend_role" "gcp" {
  role                   = "foobar"
  backend                = "gcp"
  type                   = "iam"
  bound_service_accounts = ["[email protected]"]
  token_policies         = ["bar"]
}

Output

  # resource.vault_gcp_auth_backend_role.gcp will be updated in-place
  ~ resource "vault_gcp_auth_backend_role" "gcp" {
        add_group_aliases       = false
        backend                 = "gcp"
        bound_instance_groups   = []
        bound_labels            = []
        bound_regions           = []
        bound_service_accounts  = [
            "[email protected]",
        ]
        bound_zones             = []
        id                      = "auth/gcp/role/foobar"
        max_jwt_exp             = "900"
        policies                = [
            "bar",
        ]
        role                    = "foobar"
        token_bound_cidrs       = []
        token_explicit_max_ttl  = 0
        token_max_ttl           = 0
        token_no_default_policy = false
        token_num_uses          = 0
        token_period            = 0
      ~ token_policies          = [
          + "bar",
        ]
        token_ttl               = 0
        token_type              = "default"
        type                    = "iam"
    }

Problem

No matter how many times I terraform apply, the above plan never updates. I would expect policies to be removed from the existing state and token_policies to be updated, rather than displaying a change in the plan every time.

[zx8]

Ping @lawliet89 since you were the last person to touch this resource in https://github.com/terraform-providers/terraform-provider-vault/commit/7130669d6565fbb0bb467ec7ce1a79bdc2cb7c82

[lawliet89]

token_policies require Vault >= 1.2.

In any case, after investigating, I think this is what is happening:

• When you "upgrade" from using policies to token_policies, for some reason, Terraform does not clear the value of policies list in the state to be null or empty. (https://github.com/hashicorp/terraform/issues/5290)
• This activates the code path here which is meant to support people using the older policies key when used in conjunction with Vault >=1.2

Before:

# vault_gcp_auth_backend_role.agent:
resource "vault_gcp_auth_backend_role" "agent" {
    add_group_aliases      = true
    backend                = "gcp"
    bound_instance_groups  = []
    bound_labels           = []
    bound_projects         = [
        "<SNIP>",
    ]
    bound_regions          = []
    bound_service_accounts = [
        "teamcity-agent@<SNIP>.iam.gserviceaccount.com",
    ]
    bound_zones            = []
    id                     = "auth/gcp/role/teamcity_agent"
    max_jwt_exp            = "3600"
    policies               = [
        "default",
    ]
    role                   = "teamcity_agent"
    type                   = "iam"
}

After:

resource "vault_gcp_auth_backend_role" "agent" {
    add_group_aliases       = true
    backend                 = "gcp"
    bound_instance_groups   = []
    bound_labels            = []
    bound_projects          = [
        "<SNIP>",
    ]
    bound_regions           = []
    bound_service_accounts  = [
        "teamcity-agent@<SNIP>.iam.gserviceaccount.com",
    ]
    bound_zones             = []
    id                      = "auth/gcp/role/teamcity_agent"
    max_jwt_exp             = "3600"
    policies                = [
        "default",
    ]
    role                    = "teamcity_agent"
    token_bound_cidrs       = []
    token_explicit_max_ttl  = 0
    token_max_ttl           = 0
    token_no_default_policy = false
    token_num_uses          = 0
    token_period            = 0
    token_policies          = [
        "default",
    ]
    token_ttl               = 0
    token_type              = "default"
    type                    = "iam"
}

Workaround

1. (Optional; this prevents token policies from being set incorrectly while you perform the series of steps) Set token_policies to your desired value and apply. You will now see the infinite diff.
2. Set policies to [] and apply. Omit token_policies.
3. Then, update token_policies to the desired value and apply. This should fix the issue

Solutions

• Seems like this is a Terraform bug. (https://github.com/hashicorp/terraform/issues/5290) This probably triggers because policies has Computed: true.
• Add logic in the provider to manually set policies to [] if token_policies is present during "write" time.
• (Longer term) completely remove backward support for all "old" attributes. The logic to handle both is really really messy and very painful to work with.

What do you think @kalafut and @tyrannosaurus-becks ?

[zx8]

Can confirm I'm using Vault 1.2 and that the suggested workaround resolves the issue. Thanks!

[StephenWithPH]

Adding commentary in case it helps anyone... I set the deprecated policies field rather than the recommended token_policies field, and that resolved a gnarly issue I was having (by way of Vault Agent auto-auth) with the https://github.com/hashicorp/vault-plugin-auth-gcp.

[krohmag]

I'm getting this same behavior for both token_policies and token_ttl. I don't think I can safely execute the workaround as it would strip all of my app roles of their policies, albeit briefly. For now, I'm sticking with the deprecated fields so I can plan/apply in an expected manner.

[honzous]

I'm impacted too, but my policies was already empty, so the workaround doesn't apply to me. However, I replaced token_policies with the deprecated policies and my policy is correctly attached to the role now. token_ttl and token_type fails to apply as well. I get this every time I plan after every "successful" apply:

       ~ token_ttl                       = 0 -> 3600
       + token_type                      = "service"

Vault version: 1.3.2 Terraform version: 0.12.20 Vault provider version: 2.7.1

[sharkannon]

This appears to still be an issue Vault: 1.5.5 Terraform: 0.14.5 Vault Provider: 2.18.0

Trying to get rid of our depreciation warnings, but can't because of this message. And we can't null out "policies" as that would cause an outage in production.

[kphunter]

This "infinite diff" also appears to be an issue with vault_identity_group_policies and vault_identity_group, however the workaround fixes the problem.

Vault 1.9.1 Terraform v1.1.7 on darwin_amd64

• provider registry.terraform.io/hashicorp/vault v2.24.0

[sharkannon]

Can we get a bump on this? As this still appears to be an issue, and it's kind of annoying.

[zx8]

Closing as this is 4 years old without any movement.