Destination enviroment in terraform plan/apply

[Roxyrob]

For multi account/provider configurations could be very useful to have in plan destination account (e.g. aws/azure account) to check possible error for providers configurations/issue.

[apparentlymart]

Hi @Roxyrob! Thanks for sharing this use-case.

I want to make sure we properly capture what you're hoping for here. It sounds like you'd like for the plan output to include certain information which in some way serves as global context for all of the proposed actions. You mentioned account IDs for AWS or Azure but I imagine this could also include the target region for AWS, and perhaps which IAM principal you're acting as while making the requests.

This does seem like an interesting possibility to explore. I think some questions we'd need to answer as part of designing it would be:

• How would Terraform Core know which values are interesting to display for a particular plan? The idea of an "AWS account" and similar are specific to a particular remote system, and so we typically delegate that sort of concern to individual providers, which suggests that we'd need some way for a provider to return information it expects will be interesting to the person reviewing the plan.
• What exactly would this look like in Terraform CLI's output, particular in more complex situations where there are multiple configurations for the same provider? Terraform allows declaring resources across a mix of different provider configurations for situations where objects must span across accounts and/or across regions, so we'd presumably need to make sure it's clear which information applies to which of the resources.

---

In Terraform today I believe you could approximate the effect you're looking for by including in your configuration a data resource to fetch the data you're interested in and then an output value to expose that result as part of the plan. For example, in AWS:

data "aws_caller_identity" "current" {}

data "aws_region" "current" {}

output "aws_account_id" {
  value = data.aws_caller_identity.current.account_id
}

output "aws_iam_principal" {
  value = data.aws_caller_identity.current.arn
}

output "aws_region" {
  value = data.region.current.name
}

With Terraform CLI's current UI, this should show the information as outputs in the first plan, and then show them as changing in any subsequent plan where you accidentally use a different account or region than before.

Some providers also have more explicit features aimed at similar use-cases. For example, the AWS provider has an allowed_account_ids argument whereby it'll fail configuration altogether if the selected account isn't one of the ones included in the allowed set. Rather than just showing the value on-screen for human review, this will cause the plan to fail with an explicit error message if someone accidentally uses the wrong credentials.

I'd be interested to hear if you have some success with the features I described here, and about any challenges or limitations you encounter while trying them, since that too is likely to help us to design a potential response to the use-case you shared here.

Thanks!

[Roxyrob]

Hi @apparentlymart, I'm thinking about information more tied to resources on plan output to depict what the code will do, allowing catching possible issues earlier.

One simple case I hit was a badly configured profile in credential file and code created by an identity account. Identity account manages IAM for the organization touching many accounts (naturally using different provider aliases in the same tf configuration) so some IAM roles for an account was created in another account.

In truth every time I inspect a plan I wonder... this resource will be created in the desired account (or with correct profile) ? in the desired region ? (cross the fingers)

As now I have no extended experience with tf for other providers (azure, gcp, etc) but I think the problem is common.

For a project i use data "aws_caller_identity" not for plan information but to include it as principal in policy (to be sure the deployer can access critical resources - e.g. s3 with restriction, etc.).

Clearly I think it will be better to have plan output with information linked to each resource. It is probably useful to start from a few common metadata that each provider can provide according to their needs.

[luk2302]

I would also appreciate a feature like this. The three aforementioned attributes AccountId, CallerIdentity and Region would probably be the interesting and relevant ones, with CallerIdentity probably being the least relevant one. Obviously terraform core does not know about them but the provider knows about them, similar to when an IAMRole is created terraform core does not know that it has a name and assume_role_policy, but the provider knows and lets terraform core know.

The most obvious display of that information would be to them them alongside the resource attributes. Another option would be to group the entire plan output by those attributes and first list all resources that would changed in Account 111111111111 @ eu-west-1, followed by those in 222222222222 @ us-east-1. And resource that e.g. change region or account are list as 'destroy' in one and 'create' in the other.

Is this really the first time this idea was suggested, I almost can't believe that... Maybe not many people deploy cross-region, cross-account in the same apply.

[cedricbastin]

As I undestand it Terraform uses the AWS_PROFILE env var or the profile provided in the aws provider block to determine which AWS Account it will deploy the requested resources to. It would be great to output the AWS profile & account ID, configured in the AWS config sso_account_id, at the end of a terraform plan/apply, such that it should be clearer where the shown changes will take effect and that the developer doesn't accidentally changes production when actually they wanted to test something on their test account. Thanks a lot 👍

[apparentlymart]

So far it seems like the request here is that each resource instance in the plan would separately display certain additional information about where it would be created. So perhaps that would look like this:

  # aws_instance.example will be created
  resource "aws_instance" "example" {
    ami           = "ami-whatever"
    instance_type = "m3.medium"
    subnet_id     = (known after apply)
    # ....

    region     = "us-west-2"
    account_id = "12345678"
  }

  # aws_subnet.example will be created
  resource "aws_subnet" "example" {
    cidr_block = "10.1.1.0/24"
    vpc_id     = (known after apply)

    region     = "us-west-2"
    account_id = "12345678"
  }

  # aws_vpc.example will be created
  resource "aws_vpc" "example" {
    cidr_block = "10.1.0.0/16"

    region     = "us-west-2"
    account_id = "12345678"
  }

Adding two additional lines to every resource managed by the AWS provider seems likely to make a create-time plan pretty long, and we've heard lots of times in the past that people typically find Terraform's "create" plans too long already, because they include every attribute of every resource. Therefore I expect this is something that would be useful to some (those who have complicated configurations with many regions/accounts) but annoying to others (those with one account and one region).

Therefore it seems like this would need to be more subtle than just inserting these values universally in every case. Perhaps a compromise would be to detect if there's more than one configuration for a particular provider included in the configuration, and then in only that case include an extra provider = aws.us-east-1 in the proposed change which would indicate which provider configuration each resource is associated with. You would still need to consult the configuration to learn which region and account each of your aliased provider configurations represents, but at least the connective information would be there and it would only add one additional extra row to each resource block.

---

Separately, this is also making me think again about the separate issue #16967. What these two seem to have in common is that it seems like region being a provider-configuration-level setting in the AWS provider, rather than a resource-instance-level setting, could well have been the root historical design mistake that's prompting various downstream requests to work around it.

If the AWS provider treated region and account as per-resource settings instead of as whole-provider settings (perhaps still offering a provider-level setting for a default region/account, to avoid verbosity in the simple case), as is the case with some other providers like the Google Cloud Platform provider, then the target region and account would naturally appear as part of the plan for each resource instance, because they would be attributes of the objects just like any other as far as Terraform Core is concerned.

That would end up looking like my first example above, and so the same concern about additional verbosity still applies, but it also suggests that we can use experiences with the Google Cloud Platform provider to understand whether that's a practical problem in practice: that provider is presumably already showing a region attribute for any of its regionalized resource types, so we might look to see if there have been any real complaints about the plan verbosity that implies in a configuration with many regionalized objects.

(It also isn't clear yet what exactly it might look like to change the AWS provider to support region and account on a per-resource basis, since of course the AWS provider is very large and that would be a cross-cutting change across everything. The current question is whether changing that design would have the cross-cutting benefit it seems to have on the surface, in which case it would be worth the design effort to try to see how we might get there.)

[alex-iocurrents]

It might be useful to optionally enable adding the region the resource is expected to be built in. I'm currently setting up a new AWS region and I messed up my environment files. It would have been nice to be able to double check that resources were going to be built in my expected region before I ran apply.

Moderately niche but I have run into this a few times over the last few years.

[apparentlymart]

I don't work on Terraform anymore, but FWIW the forthcoming v6.0 major release of the hashicorp/aws provider seems to be introducing the idea of tracking region on a per-resource-instance basis, instead of only globally in the provider configuration, which seems like it would deal with part of what I was hoping for in my previous comment: there is now a region argument for most resource types, and so it will presumably appear in the plan output when a new resource instance is planned for creation.

As far as I know it's still required that there be only one set of credentials per provider instance and so this would not also cause the account id to be included on a per-resource-instance basis, but it's partway there at least.