opentofu
Enable Variables in backend config block
OpenTF Version
OpenTF v1.6.0-dev
on darwin_amd64
Use Cases
Currently, if I have OpenTF configuration, I would like to use a different backend configuration when I run OpenTF for different stages, for example prod and dev AWS accounts
Today I need to use some external templating solutions such as https://github.com/kolypto/j2cli, for example -
terraform {
backend "s3" {
bucket = "terraform-state-bucket"
key = "terraform.tfstate"
region = "us-east-1"
encrypt = true
access_key = "{{ AWS_S3_BACKEND_ACCESS_KEY }}"
secret_key = "{{ AWS_S3_BACKEND_SECRET_KEY }}"
}
}
Instead, I would like not to use external tools and write this OpenTF configuration -
terraform {
backend "s3" {
bucket = "terraform-state-bucket"
key = "terraform.tfstate"
region = "us-east-1"
encrypt = true
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
}
}
And OpenTF will know how to inject the variables.
Attempted Solutions
.
Proposal
No response
References
No response
An interesting idea.
In looking at the architecture doc:
As described above, the local backend also executes operations on behalf of most other backends. It uses a state manager (either statemgr.Filesystem if the local backend is being used directly, or an implementation provided by whatever backend is being wrapped) to retrieve the current state for the workspace specified in the operation, then uses the config loader to load and do initial processing/validation of the configuration specified in the operation.
Environment variables are then more accessible at backend initialization then internal variables.
Thinking through making this work, I see three paths:
- a special variable type that gets processed first
- a pre-scan of the config to load all variables (and locals, probably)
- change the parsing order
Of them, 3. seems the most complex. 1. would enhance the language and so I would expect much discussion around the design. 2. seems the most immediately practical approach.
In terms of the example use case this risks introducing a security footgun, potentially increasing the likelihood of credentials being stored in tfvars files.
Today I need to use some external templating solutions such as https://github.com/kolypto/j2cli, for example
The backend is already capable of reading credentials from the standard AWS env vars. In terms of the given use case, the Jinja processing step is not required and adds unnecessary complexity to achieve something that already works out of the box.
a pre-scan of the config to load all variables (and locals, probably)
@Hefeweizen Locals could be problematic as they can reference resource attributes which can not be known without loading state.
IMO options 2 and 3 are of equivalent complexity, and option 1 (#450) would be the cleanest way to implement this.
In terms of the example use case this risks introducing a security footgun, potentially increasing the likelihood of credentials being stored in tfvars files.
Today I need to use some external templating solutions such as https://github.com/kolypto/j2cli, for example
The backend is already capable of reading credentials from the standard AWS env vars. In terms of the given use case, the Jinja processing step is not required and adds unnecessary complexity to achieve something that already works out of the box.
a pre-scan of the config to load all variables (and locals, probably)
@Hefeweizen Locals could be problematic as they can reference resource attributes which can not be known without loading state.
IMO options 2 and 3 are of equivalent complexity, and option 1 (#450) would be the cleanest way to implement this.
I agree that for the credentials parameters we already have the options of using environment variables, however, I see a huge benefit on being able to declare things such as the bucket and region as variables. At the moment, providing values to the backend can only be done by using a workaround (terraform init -backend-config foo.tf, the foo.tf file containing the values in a key=value format). I have to admit that implementing this no matter which option, sounds quite complex though.
We'd like to explore ways of doing this, so marking this as accepted, but it needs to go through the RFC process to actually pick a solution. Right now there is a related RFC that would solve this: https://github.com/opentofu/opentofu/issues/1042
