Inconsistent behaviour when specifying `subscription_id` attribute in provider block for new Subscriptions

[krowlandson]

Is there an existing issue for this?

• [X] I have searched the existing issues

Community Note

• Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

1.2.7

AzureRM Provider Version

3.18.0

Affected Resource(s)/Data Source(s)

azurerm_subscription

Terraform Configuration Files

# Base provider, used to read the billing account and create the new Subscription
provider "azurerm" {
  features {}
}

# Data resource to get the billing_scope value
data "azurerm_billing_enrollment_account_scope" "example" {
  billing_account_name    = local.billing_account_name
  enrollment_account_name = local.enrollment_account_name
}

# Create the new Subscription (uses the Microsoft.Subscription/Aliases provider)
resource "azurerm_subscription" "example" {
  subscription_name = "My Test Subscription"
  alias             = "new-sub-test-001"
  billing_scope_id  = data.azurerm_billing_enrollment_account_scope.example.id
}

# Define a secondary provider with an alias, configured to target the newly created Subscription
provider "azurerm" {
  alias           = "new_sub"
  subscription_id = azurerm_subscription.example.subscription_id
  features {}
}

# Create a Resource Group in the newly created Subscription
resource "azurerm_resource_group" "example" {
  provider = azurerm.new_sub
  name     = "new-sub-test"
  location = "West Europe"
}

Debug Output/Panic Output

# see below

Expected Behaviour

When trying to create a new Azure Subscription and then cross-referencing the Subscription ID of the new Subscription into a provider block we expect to see consistent behaviour regardless of the authentication method used.

Actual Behaviour

When authenticating with a Service Principal the initial terraform plan throws the following error:

╷
│ Error: building AzureRM Client: 1 error occurred:
│       * A Subscription ID must be configured when authenticating as a Service Principal using a Client Secret.
│
│
│
│   with provider["registry.terraform.io/hashicorp/azurerm"].new_sub,
│   on main.tf line 20, in provider "azurerm":
│   20: provider "azurerm" {
│
╵

We believe this to be correct, as at this point the new Subscription ID is unknown and therefore cannot be correctly set on the provider.

When we authenticate using the Azure CLI the initial terraform plan is successful, but we get the following error during terraform apply:

╷
│ Error: building AzureRM Client: obtain subscription(00e960fd-76a5-410b-9391-877383a5974f) from Azure CLI: parsing json result from the Azure CLI: waiting for the Azure CLI: exit status 1: ERROR: Subscription '00e960fd-76a5-410b-9391-877383a5974f' not found. Check the spelling and casing and try again.
│ 
│   with provider["registry.terraform.io/hashicorp/azurerm"].new_sub,
│   on main.tf line 20, in provider "azurerm":
│   20: provider "azurerm" {
│
╵

We believe in this case that the provider is incorrectly accepting a null or empty string input on the subscription_id attribute of the provider block and instead defaulting to the current default Subscription associated with the Azure CLI authenticated user.

This is incorrectly giving customers the illusion that they can create a new Subscription and then deploy resources straight into it within a single apply of a single root module.

Steps to Reproduce

1. Create a configuration matching the example, providing valid values for access to the billing account for Subscription create
2. Authenticate using a Service Principal with suitable permissions
3. terraform init (should work)
4. terraform plan (should throw error)
5. Authenticate using Azure CLI with suitable permissions
6. terraform init (should work)
7. terraform plan (should work)
8. terraform apply (should create new Subscription and then throw error)
9. terraform apply (should throw the following error)
10. az account list --refresh (to refresh the list of available Subscriptions)
11. May need to run az account set to select a subscription
12. terraform apply (should work the second time as the Subscription now exists!)

Important Factoids

n/a

References

I believe this issue identifies the underlying reason why customers are trying to create a new Subscription and then deploy resources to it, only to face issues which surface as issues such as:

• #14863
• #17541

We have successfully worked around this situation with our lz-vending module, but this was only made possible as the new AzAPI provider has an option to set the parent_id for any resource being created,

We believe the addition of a new optional parent_id argument for all resources in the azurerm provider would allow customers to deploy resources across multiple Subscriptions without hitting this issue (or the need to provide multiple aliased providers).