Reload client configuration periodically for remote state storage

[juljaeg]

Terraform Version

1.11.4

Use Cases

We are refreshing the token in the file periodically and it would be nice for terraform to pick up changes in the file while it is running or reread it from time to time or close to expiry. We are using terraform in azure devops pipelines and the oidc tokens expire after 10 minutes. We are using multiple service connections within one apply which looks not possible with the new azure devops implementation. For applies under 10 minutes and kubernetes with kubelogin for AKS this works very well as kubelogin is invoked more often and reads the file when the process starts.

Attempted Solutions

We looked into the service connection id based approach which seems to not support multiple service connections.

Proposal

No response

References

No response

[jbardin]

Hi @juljaeg,

Can you explain what the client_token_file_path refers to, or what subsystem you expect Terraform to be reloading? This sounds like a feature request for the AzureRM provider, or maybe the azure state storage, but I'm not sure from the information provided.

[juljaeg]

Hi @jbardin,

thanks for the quick response. I am mainly refering to this line:

https://github.com/hashicorp/terraform/blob/6d6352220b5fee5f0c2c0f894f41d59d7e6f44d7/internal/backend/remote-state/azure/backend.go#L292

This calls the helper which either reads the field or reads the token from the files content referenced by client_token_file_path. I took a look at hashicorp/go-azure-sdk and there seems at least a fallback mechanism based on the token expiry here:

https://github.com/hashicorp/go-azure-sdk/blob/e69969765468264aac1f33333f6d93887c816327/sdk/auth/cached_authorizer.go#L30C2-L32C19

I am not sure how and if this cached client implementation is used here. I raised the issue here as the code reading from the file is here 😄

[jbardin]

Thanks @juljaeg, that does make it a Terraform issue, though it's not really something we can change easily. The interface is designed in a generic way such that the state storage client is configured once for all types, and there is no hook call Configure again, which could entail throwing away the entire context and starting over.

We can leave this here as something to consider for a new state storage subsystem, which will probably be executed more like providers. That doesn't make the problem easier however, but there is at least new design work being done in that area.

[jbardin]

I knew we had something similar, so linking to #32100. While closely related, we can keep this is the state-storage version of the provider credential issue.

[SarahFrench]

Also, cross-linking to https://github.com/hashicorp/terraform-provider-azurerm/issues/29799 in the AzureRM provider's issues, as work is likely to be re-used across both.

[magodo]

@juljaeg Since 1.11.2, instead of using the oidc token file path when running in ADO pipeline, you can follow https://developer.hashicorp.com/terraform/language/backend/azurerm#example-configuration-for-azure-devops to use the ADO Pipelines OIDC auth via OIDC Request Token + URL + Service Connection ID, which will refresh the OIDC token if expired.