Azure backend with OIDC auth seems broken in version 1.11.0

[audunsolemdal]

Terraform Version

1.11.0

Terraform Configuration Files

providers.tf

  backend "azurerm" {
    subscription_id      = xxx
    resource_group_name  = yyy
    storage_account_name = zzz
    container_name       = ...
    key                  = "terraform.tfstate"
    use_azuread_auth     = true
  }
}

env vars set in my github actions workflow:

  ARM_CLIENT_ID: ${{vars.AZURE_OIDC_CLIENT_ID}}
  ARM_USE_OIDC: true
  ARM_TENANT_ID: ${{vars.TENANT_ID}}

Debug Output

Error: retrieving Storage Account (Subscription: "xxxxxxxx"
Resource Group Name: yyyyyy"
Storage Account Name: "aaaaa"): unexpected status 403 (403 Forbidden) with error: AuthorizationFailed: The client '.... does not have authorization to perform action 'Microsoft.Storage/storageAccounts/read' over scope `resourceid to storage account `or the scope is invalid. If access was recently granted, please refresh your credentials.

Expected Behavior

OIDC authentication towards azurerm backend should work without breaking changes

Actual Behavior

Seems that this PR broke OIDC auth towards azurerm backends(?) and the release notes do not mention breaking changes. https://github.com/hashicorp/terraform/pull/36258

It is not immediately obvious how to fix this apart from downgrading to version 1.10.x

Steps to Reproduce

terraform init

Additional Context

No response

References

No response

Generative AI / LLM assisted development?

No response

[r-poirier]

We also encoutered this error with the upgrade. Giving the identity accessing the backend the Reader role on the storage account seems to fix the issue.

[audunsolemdal]

We also encoutered this error with the upgrade. Giving the identity accessing the backend the Reader role on the storage account seems to fix the issue.

Interesting, in our current setup we only grant Storage Blob Data Owner to managed identities at the blob container level, no assignments at the storage account level. Seems to be a regression with regards to permissions which hopefully can be improved?

[audunsolemdal]

Semi-related, would be interesting to see if this issue is resolved by the mentioned PR https://github.com/hashicorp/terraform/issues/28423

[OskarKlintrot]

I seem to have hit this as well. I couldn't for the life of me figure out what I had changed until I noticed I had a new version of TF. If I revert back to 1.10.5 everything works as before.

[iantriggs]

Exact same error here, rolling back to 1.10 has worked for me too.

[adelinn]

Same issue here. For some reason the azurerm backend now requires an additional permission to work, Microsoft.Storage/storageAccounts/read on the whole storage account. This new permission is not strictly needed and shouldn't be required for the backend to work.

[jaredfholgate]

@magodo It looks like an extra read has been introduced as part of the backend updates. Previously the minimum permissions required were Storage Blob Data Owner on the container when use_azuread_auth was set. It is now asking for read access to the whole storage account and breaking many existing implementations. I'll try to take a look at the code and see where it is happening.

In the short term we are pinning to 1.10.5 where we can.

[jaredfholgate]

@magodo It looks like an extra read has been introduced as part of the backend updates. Previously the minimum permissions required were Storage Blob Data Owner on the container when use_azuread_auth was set. It is now asking for read access to the whole storage account and breaking many existing implementations. I'll try to take a look at the code and see where it is happening.

In the short term we are pinning to 1.10.5 where we can.

@magodo Looks like we may be missing a return here? https://github.com/hashicorp/terraform/blob/473e5931ac76948cfa2bdf9ea9184bdb2db4b116/internal/backend/remote-state/azure/api_client.go#L73

Looks like it is trying to query the storage account meta data here: https://github.com/hashicorp/terraform/blob/473e5931ac76948cfa2bdf9ea9184bdb2db4b116/internal/backend/remote-state/azure/api_client.go#L88

I'm guessing it only needs that client.accountDetail when getting a SAS, which is not required for Entra ID auth?

Update: I now see the comment on the PR: One implementation detail is that the using the same Azure storage dataplane SDK, the storage client requires a base URI of the storage account, which is derived by sending a GET to the storage account. This is skipped in case the storage shared access key or sas token is specified, which is to behave identically as the current version.

This appears to be the root cause problem. Hopefully there is a way to derive the URI without needing to query the end point?

PR: https://github.com/hashicorp/terraform/pull/36258

[magodo]

Sorry * for the breaking change.. I've made a fix in #36623, and tested it as much as possible to ensure all the cases are covered (see the PR description).

@audunsolemdal About your specific case, you'll probably have to remove resource_group_name from your configuration even after #36623 is merged. This is to allow users who uses private DNS zone in the storage account can get the blob endpoint via a management plane API call, which is indicated by the precense of resource_group_name and subscription_id.

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.