hashicorp
Feature Request: Opt-in support for ephemeral values in Terraform state
Terraform Version
latest
Use Cases
Currently, Terraform disallows ephemeral values in non-write-only attributes. However, many resources produce a mix of values, some that should not be stored in state (ephemeral, often sensitive), and others that must persist in state to be useful.
Examples:
-
tls_private_key- Private key PEM → should remain ephemeral (used at apply time only).
- Public key → often needs to stick around in the state and be passed around to various places.
-
vault_kv_secret_v2- Secret data payload → should remain ephemeral, never written to state.
- Metadata (e.g., version) → often useful for triggers or referencing in other resources, but currently cannot persist without persisting the entire secret block.
Without a way to selectively mark attributes as ephemeral vs. persistable, users are forced into unsafe or awkward workarounds.
Attempted Solutions
- Writing secrets into local files with null_resource and local-exec.
- Re-fetching public/metadata values with external data sources
- Accepting that secrets leak into the state file
- Generating keys by hand/externally
Proposal
Introduce a mechanism to opt in to persisting only selected values in state while treating others as ephemeral.
Two possible designs:
Design 1: New function: nonephemeral()
Mark an expression as explicitly allowed to persist in state, while sibling attributes remain ephemeral.
ephemeral "tls_private_key" "deploy" {
algorithm = "ED25519"
}
# Persist public key, do not persist private key
resource "google_compute_instance" "instance" {
...
metadata.ssh-keys= nonephemeral(ephemeral.tls_private_key.deploy.public_key_openssh)
}
Vault example:
ephemeral "vault_kv_secret_v2" "cfg" {
mount = "kv"
name = "app/config"
}
# Persist only metadata.version
resource "google_compute_instance" "instance" {
metadata.config_version = nonephemeral(ephemeral.tls_private_key.deploy.public_key_openssh)
}
Design 2: terraform_data opt-in
Make terraform_data to explicitly allow persistence of ephemeral values:
ephemeral "tls_private_key" "deploy" {
algorithm = "ED25519"
}
resource "terraform_data" "ssh_pubkey" {
input = {
pubkey = ephemeral.tls_private_key.deploy.public_key_openssh
}
}
Vault example:
ephemeral "vault_kv_secret_v2" "cfg" {
mount = "kv"
name = "app/config"
}
resource "terraform_data" "ssh_pubkey" {
input = {
pubkey = ephemeral.tls_private_key.cfg.version
}
}
References
No response
This becomes extra neccesary if providers follow the trend of data "vault_kv_secret_v2", which is deprecated.
The requirement that ephemeral values not be used in "normal" resource arguments is primarily a technical constraint to meet the requirements of the rest of the system: Terraform's plan/apply flow relies on all non-ephemeral values remaining constant throughout an entire plan/apply round unless they are "unknown" during the planning phase, because otherwise any derived values could also change and thus significantly change the plan.
However, I think this could potentially work if it included the additional constraint that this new nonephemeral function always returns an unknown value during the planning phase, so that the plan-time value is effectively ignored and only the apply-time value gets saved in the state.
If that were true then perhaps it would be clearer to name it something like applytime, intended to communicate that this function returns the "apply-time value" of its argument:
ephemeral "tls_private_key" "deploy" {
algorithm = "ED25519"
}
# Persist public key, do not persist private key
resource "google_compute_instance" "instance" {
...
metadata = {
ssh-keys = applytime(ephemeral.tls_private_key.deploy.public_key_openssh)
}
}
I would imagine this then causing that ssh-keys element to be shown as (known after apply) in the plan output[^1][^2], and then whatever public key was generated during the apply phase (as a non-ephemeral value) would be used as the final value for that metadata key.
That could potentially be combined with count = terraform.applying ? 1 : 0 in the ephemeral block to completely skip generating a key during the planning phase. That isn't really important for tls_private_key in particular since it's only doing local computation anyway, but would perhaps be more useful for the likes of vault_kv_secret_v2 where it would avoid making a redundant network request to the Vault server and avoid the plan phase needing access to the secret. (Whether it would be a good idea to copy a leased Vault secret to persistent storage outside of Vault is a different question, of course.)
[^1]: This is a little like the function timestamp, which is defined to return the the time of the apply phase and therefore returns an unknown value during the planning phase because Terraform can't predict when the apply phase will run. If these features had been developed in a different order then maybe timestamp would've been defined to return an ephemeral value and then applytime(timestamp()) would produce a non-ephemeral value describing the time of the apply phase.
[^2]: This applytime function could also be used in conjunction with functions like file to force the file to be read only during the apply phase in cases where the file might not exist yet during planning, or might contain a stale value from a previous run. applytime(file("${path.module}/whatever.txt")).
Thanks @apparentlymart, I was thinking along much the same lines here. The major incompatibility is really just with the value changing between plan and apply, so forcing it be unknown works, and like you said, we already do this with timestamp. I've been toying with different ideas like this also for user debugging scenarios, since the guarantees of not serializing ephemeral data are odds with troubleshooting the config evaluation, though of course debugging covers other use cases unrelated to ephemeral values too.
The tricky part here however is that just like timestamp, applytime would result in a change during every single plan (though some cases could be supressed with ignore_changes). I don't know if there's a good way around that, because Terraform must assume that the value can change every time it's evaluated. I'm not sure how we'd also incorporate count = terraform.applying ? 1 : 0, for the same reasons why we made terraform.applying ephemeral in the first place, the number of instances would change between plan and apply, also affecting all references to those instances.
There may be some promise in further utilizing terraform_data to assign a different lifecycle to the ephemeral data (giving arbitrary data a resource lifecycle was the basis of the idea for that resource). I think the use cases would take multiple different attributes to cover, but maybe something could be designed which doesn't end up being too fiddly.
Agreed with all of the tradeoffs about applytime returning unknown. FWIW this is the shape of using terraform.applying that I was imagining:
ephemeral "tls_private_key" "deploy" {
count = terraform.applying ? 1 : 0
algorithm = "ED25519"
}
# Persist public key, do not persist private key
resource "google_compute_instance" "instance" {
...
metadata = {
ssh-keys = applytime(one(ephemeral.tls_private_key.deploy[*].public_key_openssh))
}
}
The reference expression could be less clunky with something like https://github.com/hashicorp/terraform/issues/21953 of course, but that's an entirely separate can of worms. 🙃
While writing this out I also remembered that my earlier experiment with apparentlymart/memory's memory resource type could also maybe serve as a workaround in the meantime.
Something like this? 🤷♂️ (not actually tested)
variable "reset_ssh_key" {
type = bool
default = false
}
ephemeral "tls_private_key" "deploy" {
count = var.reset_ssh_key && terraform.applying ? 1 : 0
algorithm = "ED25519"
}
resource "memory" "ssh_public_key" {
new_value = one(ephemeral.tls_private_key.deploy[*].public_key_openssh)
}
# Persist public key, do not persist private key
resource "google_compute_instance" "instance" {
...
metadata = {
ssh-keys = memory.ssh_public_key.value
}
}
This provider's trickery also addresses the potential annoyance of the value always being unknown during planning: in this case it'd be unknown only when var.reset_ssh_key is true, which is also the only time when the write-only attributes (not shown) would need to be updated.
It's very, very clunky but I think it might actually work with today's Terraform, without any new language features. 🤔
If this approach seems to work then a resource type like memory could potentially live in the terraform.io/builtin/terraform provider, so it'd be available without installing any external plugins.
Another way to solve the permadiff issue might to "bundle" every ephemeral value with some (non ephemeral) "metadata". Providers would have to guarantee that the metadata value is updated if-and-only-if the ephemeral value has changed. For vault this could be the secret version, for tls_private_key it might be the "creation datetime".
Terraform plan would then know if the ephemeral value (apply time) will change, by knowing whether the "metadata" has changed (plan time).
To prevent race conditions leading to apply failures, this metadata should also serve as a lookup key. During apply terraform should then lookup the "ephemeral" value that is identified by the metadata (only relevant when the generation happens externally, e.g. "vault").
A benefit is that this would integrate with attributes like kv_secret_v2#data_json_wo_version that also aim to solve plannability with ephemeral values
A downside with a solution like this is that it requires provider changes.
@apparentlymart, ha, that is similar to what I was thinking about adding to terraform_data, we just didn't have a common use case. Having a dedicated structure in there for the various known ephemeral workflows (ephemeral that's effectively static but should be replaced when it differs, ephemeral that you need to capture once then rotate on some other change, etc), would make it a little less ambiguous about what it's doing, but isn't really any loss clunky. Maybe that's OK though, because the need to bypass the ephemerality shouldn't be a primary use case?
@rwblokzijl, that would greatly change the implementation of ephemeral, requiring a different protocol, effectively deprecating existing provider releases. Changes to providers are decidedly non-trivial. This is why I would lean toward leveraging the tools we have on hand, vs rethinking what it means to be ephemeral.
Just started to test out the ephemeral feature, and was trying the following:
locals {
ed25519_secret_key_hash = sha1(var.ed25519_secret_key)
}
resource "aws_ssm_parameter" "ed25519_secret_key" {
name = "${var.name}-ed25519-secret-key"
type = "SecureString"
value_wo = var.ed25519_secret_key
value_wo_version = parseint(substr(local.ed25519_secret_key_hash, 0, 8), 16)
}
Where ed25519_secret_key is an ephemeral variable.
Conceptually it makes total sense to do this, but it doesn't work.
Error: Invalid use of ephemeral value
with module.eu-central-1-node-operator-1.aws_ssm_parameter.ed25519_secret_key
on ../modules/node-operator/main.tf line 42, in resource "aws_ssm_parameter" "ed25519_secret_key":
value_wo_version = parseint(substr(local.ed25519_secret_key_hash, 0, 8), 16)
Ephemeral values are not valid for "value_wo_version", because it is not a write-only attribute and must be persisted to state.
I thought "maybe there's a similar thing to nonsensitive", but there isn't one.
So, the only way to handle this use-case right now is to either manually handle the version, or do it outside terraform?
FWIW, I think that last comment overlaps with https://github.com/hashicorp/terraform-provider-aws/issues/44590 , which is describing a similar concern from a provider perspective.
Can I ask if there is a stable work around for this issue, such that a public key can be read from an ephemeral tls_private_key?
I have a situation like that of ephemeral.tls_private_key.deploy[*].public_key_openssh in @apparentlymart's example above but am not able to synthesise a workaround from the material on this page.
