terraform-provider-azapi icon indicating copy to clipboard operation
terraform-provider-azapi copied to clipboard

Ignores Az Login credentials and uses MSI

Open saulobejo opened this issue 1 year ago • 2 comments

After logging-in using az cli, azapi keeps using MSI for requests, failing to access the API because of that.

Terraform version: 1.3.2 registry.terraform.io/azure/azapi v1.0.0 registry.terraform.io/hashicorp/azurerm v3.26.0

terraform code (piece)

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.26.0"
    }
    null = {
      source = "hashicorp/null"
      version = "~> 3.1.1"
    }

    local = {
      source = "hashicorp/local"
      version = "~> 2.2.3"
    }

    azapi = {
      source = "azure/azapi"
      version = "~> 1.0.0"
    }
  }

  required_version = ">= 1.3.2"

  backend "http" {}
}

provider "azurerm" {
  features {}

  subscription_id = var.subscription_id
  tenant_id       = var.tenant_id
}

provider "azapi" {
  subscription_id = var.subscription_id
  tenant_id       = var.tenant_id
}


data "azapi_resource" "agw" {
  name      = var.agw_name
  parent_id = data.azurerm_resource_group.agw_rg.id
  type      = "Microsoft.Network/applicationGateways@2016-03-30"

#   response_export_values = ["properties.loginServer", "properties.policies.quarantinePolicy.status"]
}

AZ REST works without problems

az rest --url https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.Network/applicationGateways/AgName?api-version=2022-05-01

Terraform plan fails

2022-10-12T22:25:26.734Z [DEBUG] provider.terraform-provider-azapi_v1.0.0: Oct 12 22:25:26.734145 Retry: 
=====> Try=1 GET https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.Network/applicationGateways/AgName?api-version=2016-03-30: timestamp=2022-10-12T22:25:26.734Z
2022-10-12T22:25:26.734Z [DEBUG] provider.terraform-provider-azapi_v1.0.0: Oct 12 22:25:26.734330 Retry: 
=====> Try=1 GET http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fmanagement.core.windows.net%2F: timestamp=2022-10-12T22:25:26.734Z
2022-10-12T22:25:26.734Z [DEBUG] provider.terraform-provider-azapi_v1.0.0: Oct 12 22:25:26.734448 Request: ==> OUTGOING REQUEST (Try=1)
   GET http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=REDACTED
   Metadata: REDACTED
   User-Agent: azsdk-go-azidentity/v1.0.0 (go1.18.2; linux): timestamp=2022-10-12T22:25:26.734Z
2022-10-12T22:25:26.747Z [DEBUG] provider.terraform-provider-azapi_v1.0.0: Oct 12 22:25:26.747304 Response: ==> REQUEST/RESPONSE (Try=1/12.701129ms, OpTime=12.80493ms) -- RESPONSE RECEIVED
   GET http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=REDACTED
   Metadata: REDACTED
   User-Agent: azsdk-go-azidentity/v1.0.0 (go1.18.2; linux)
   --------------------------------------------------------------------------------
   RESPONSE Status: 200 OK
   Content-Length: 1748
   Content-Type: application/json; charset=utf-8
   Date: Wed, 12 Oct 2022 22:25:25 GMT
   Server: IMDS/150.870.65.684: timestamp=2022-10-12T22:25:26.747Z
2022-10-12T22:25:26.747Z [DEBUG] provider.terraform-provider-azapi_v1.0.0: Oct 12 22:25:26.747600 Retry: response 200: timestamp=2022-10-12T22:25:26.747Z
2022-10-12T22:25:26.748Z [DEBUG] provider.terraform-provider-azapi_v1.0.0: Oct 12 22:25:26.748093 Authentication: *azidentity.ManagedIdentityCredential.GetToken() acquired a token for scope https://management.core.windows.net//.default: timestamp=2022-10-12T22:25:26.748Z
2022-10-12T22:25:26.748Z [DEBUG] provider.terraform-provider-azapi_v1.0.0: Oct 12 22:25:26.748231 Authentication: DefaultAzureCredential authenticated with ManagedIdentityCredential: timestamp=2022-10-12T22:25:26.748Z
2022-10-12T22:25:26.749Z [DEBUG] provider.terraform-provider-azapi_v1.0.0: Oct 12 22:25:26.749084 Request: ==> OUTGOING REQUEST (Try=1)
   GET https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.Network/applicationGateways/AgName?api-version=2016-03-30
   Accept: application/json
   Authorization: REDACTED
   User-Agent: HashiCorp Terraform/1.3.2 (+https://www.terraform.io) Terraform Plugin SDK/2.8.0 terraform-provider-azapi/v1.0.0 pid-222c6c49-1b0a-5959-a213-6608f9eb8820
   X-Ms-Correlation-Request-Id: REDACTED
   Request contained no body: timestamp=2022-10-12T22:25:26.749Z
2022-10-12T22:25:26.850Z [DEBUG] provider.terraform-provider-azapi_v1.0.0: Oct 12 22:25:26.850367 Response: ==> REQUEST/RESPONSE (Try=1/101.083219ms, OpTime=101.225522ms) -- RESPONS
E RECEIVED
   GET https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.Network/applicationGateways/bejo-int-a
g?api-version=2016-03-30
   Accept: application/json
   Authorization: REDACTED
   User-Agent: HashiCorp Terraform/1.3.2 (+https://www.terraform.io) Terraform Plugin SDK/2.8.0 terraform-provider-azapi/v1.0.0 pid-222c6c49-1b0a-5959-a213-6608f9eb8820
   X-Ms-Correlation-Request-Id: REDACTED
   --------------------------------------------------------------------------------
   RESPONSE Status: 403 Forbidden
   Cache-Control: no-cache
   Content-Length: 498
   Content-Type: application/json; charset=utf-8
   Date: Wed, 12 Oct 2022 22:25:26 GMT
   Expires: -1
   Pragma: no-cache
   Strict-Transport-Security: REDACTED
   X-Content-Type-Options: REDACTED
   X-Ms-Correlation-Request-Id: REDACTED
   X-Ms-Failure-Cause: REDACTED
   X-Ms-Request-Id: 3f723329-df5e-49d3-bdd3-726628e37f5f
   X-Ms-Routing-Request-Id: REDACTED
   --------------------------------------------------------------------------------
{"error":{"code":"AuthorizationFailed","message":"The client 'yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy' with object id 'yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy' does not have authorization
 to perform action 'Microsoft.Network/applicationGateways/read' over scope '/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/ResourceGroupName/providers/Microso
ft.Network/applicationGateways/AgName' or the scope is invalid. If access was recently granted, please refresh your credentials."}}
   --------------------------------------------------------------------------------: timestamp=2022-10-12T22:25:26.850Z
2022-10-12T22:25:26.850Z [DEBUG] provider.terraform-provider-azapi_v1.0.0: Oct 12 22:25:26.850730 Retry: response 403: timestamp=2022-10-12T22:25:26.850Z
  │ Error: reading "Resource: (ResourceId \"/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.Network/applicationGateways/AgName\" / Api Version \"2016-03-30\")": GET https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.Network/applicationGateways/AgName
  │ --------------------------------------------------------------------------------
  │ RESPONSE 403: 403 Forbidden
  │ ERROR CODE: AuthorizationFailed
  │ --------------------------------------------------------------------------------
  │ {
  │   "error": {
  │     "code": "AuthorizationFailed",
  │     "message": "The client 'yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy' with object id 'yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy' does not have authorization to perform action 'Microsoft.Network/applicationGateways/read' over scope '/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/resourceGroups/ResourceGroupName/providers/Microsoft.Network/applicationGateways/AgName' or the scope is invalid. If access was recently granted, please refresh your credentials."
  │   }
  │ }
  │ --------------------------------------------------------------------------------
  │ 
  │ 
  │   with data.azapi_resource.agw,
  │   on gateway.api.tf line 13, in data "azapi_resource" "agw":
  │   13: data "azapi_resource" "agw" {

with XXXX being the subscription ID YYYY being the MSI client ID (which should not be used!)

The expectation is that it would use the current user logged with az cli which, as the `az rest' command shows, works and has access.

saulobejo avatar Oct 12 '22 22:10 saulobejo

Hi @saulobejo ,

Thank you for taking time to report this issue and provide the detailed logs!

The MSI has higher priority than azure cli login. Please check if there're environment variables like AZURE_CLIENT_SECRET, ARM_CLIENT_ID, and remove them.

ms-henglu avatar Oct 13 '22 02:10 ms-henglu

@ms-henglu thanks for the quick reply. Maybe instead of depending on priority azpi could use use_msi as azurerm does.

printenv | sort | grep AZURE_ and printenv | sort | grep ARM_ returns no variables with such prefixes in the environment.

saulobejo avatar Oct 13 '22 07:10 saulobejo

I am having the same issue. This is different behaviour from the azurerm provider, which this provider is meant to compliment. For us this adds the headache of having to source the client id solely for this purpose. That is a huge hurdle currently.

Any news on this?

hermannolafs avatar Nov 22 '22 09:11 hermannolafs

Since azapi uses Azure Go SDK to complete token authentication which is different from azurerm, we need Azure Go SDK to expose an interface to us to make msi selectable. A feature request has been submitted to Azure Go SDK.

ms-zhenhua avatar Jan 12 '23 09:01 ms-zhenhua

Is there any workaround for this? It looks like the Azure Go SDK won't be updated until April. Due to private tfstates we need the ability to run terraform via a jumphost (Azure VM running Ubuntu). But I can't see a way to force azapi to use the cli logged in user rather than the jumphost identity. I'd rather not use a service principal here

djw-bsn avatar Jan 31 '23 08:01 djw-bsn

I am getting the following error on a local machine with no instance metadata:

DefaultAzureCredential authentication failed
│ GET http://169.254.169.254/metadata/identity/oauth2/token
│ --------------------------------------------------------------------------------
│ RESPONSE 403 connecting to 169.254.169.254:80: connecting to 169.254.169.254:80: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network.
│ --------------------------------------------------------------------------------
│ connecting to 169.254.169.254:80: connecting to 169.254.169.254:80: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network.
│ --------------------------------------------------------------------------------

Is there a way to skip the instance metadata check when using the local az cli. I am connected with my azure user account.

didayal-msft avatar Apr 04 '23 11:04 didayal-msft