Variable validation `error_message` dynamic evaluation fails

[KyleKotowick]

Terraform Version

Terraform v1.9.0
on windows_amd64

Terraform Configuration Files

/module/variables.tf

variable "error_message" {
  type     = string
  nullable = false
}

variable "test_input" {
  type = bool
  validation {
    condition     = var.test_input == true
    error_message = var.error_message
  }
}

/main.tf

module "demo" {
  source        = "./module"
  test_input    = false
  error_message = "sample error"
}

Debug Output

https://gist.github.com/KyleKotowick/e7d872d6019409b08fe457b3d026003a

Expected Behavior

The terraform plan should have failed with the error message:

│ Error: Invalid value for variable │ │ on main.tf line 3, in module "demo": │ 3: test_input = false │ ├──────────────── │ │ var.test_input is false │ │ sample error │ │ This was checked by the validation rule at module\variables.tf:8,3-13.

Actual Behavior

Terraform failed to evaluate the dynamic variable validation error message, and instead used "Failed to evaluate condition error message.":

│ Error: Invalid value for variable │ │ on main.tf line 3, in module "demo": │ 3: test_input = false │ ├──────────────── │ │ var.test_input is false │ │ Failed to evaluate condition error message. │ │ This was checked by the validation rule at module\variables.tf:8,3-13.

Steps to Reproduce

1. terraform init
2. terraform plan

Additional Context

No response

References

No response

[apparentlymart]

Hi @KyleKotowick! Thanks for reporting this.

Based on the trace log I see that this is failing in the validation phase rather than in the planning phase. terraform plan implicitly recreates the effect of terraform validate before doing any other work, and Terraform tries to evaluate variable validation rules during the validation phase when possible.

"When possible" means that if condition returns an unknown value then Terraform will optimistically continue on the assumption that the result will become known either during the planning phase or, in the worst case, during the apply phase.

However, this is an interesting case where the condition is known to be false -- i.e. the validation has failed -- but the error message is an unknown value, because it refers to another input variable and other input variables become known only during the planning phase once global validation has become possible. Terraform is trying its best to report that the condition failed during validation, even though it doesn't have a known error message to report at that phase.

Although this particular case shows it happening with secondary input variables, this problem would arise if error_message referred to anything that's unknown during the validation phase, which also includes data source results, managed resource result attributes, and a few other quirky situations like the small number of impure built-in functions.

When it was only possible for condition and error_message to refer to the specific value being validated there was never any situation where the condition could be known but the error_message unknown: if the condition were not known then the error message would not get evaluated at all. The only exception was that we did previously allow calling impure functions like timestamp in error_message, but it seems like nobody thought to try that before or they would've caused this same error message.

Now that we support referring to other objects in both arguments, we're going to need to decide how we should treat the situation where the condition is unknown but the error_message is not. One possible compromise that comes to mind is:

• If and only if we're currently in the validation phase, if error_message turns out to be unknown when we want to report a validation error then we skip reporting it so that we can continue to the plan phase where there will be more information available.
• It's also possible for that situation to arise during the planning phase, but it's less likely. I imagine we'd prefer to return a new error message saying that the error message is unknown if this arises during the plan phase, rather than ignoring the known error until the apply phase where we might detect the problem only after some changes have been made.
• It should never be possible for either condition or error_message to be unknown during the apply phase, so no behavior change is needed for the apply phase.

While we're evaluating that idea, I'll see about implementing a more minimal temporary fix of having a special error message for when error_message turns out to be unknown even during the validation phase, since that would then be only a cosmetic change to improve the error message. I don't think that should be the final fix, but will at least give better feedback about what the problem is in the meantime while we discuss the final solution.

Thanks again for reporting this.

[KyleKotowick]

@apparentlymart I'm not quite following why condition is known to be false, but error_message is an unknown value. Both condition and error_message require the input variables, and if input variables become known only during the planning phase, then neither condition nor error_message should be known until the planning phase. How can condition be known to be false during validation, since it depends on an input variable?

we're going to need to decide how we should treat the situation where the condition is unknown but the error_message is not.

I think this is backwards of the scenario you describe; you said the issue arose because condition is known but error_message is not.

Since the error message is important, which I imagine is the reason you've implemented support for dynamic error messages, I would vote for any option that causes the intended error message to be output. Otherwise, you're defeating the purpose of having the error messages.

[apparentlymart]

Hi @KyleKotowick,

In general, input variables are always unknown during validation. Due to a historical implementation mistake, the variable validation rules ended up being able to run during the validation phase only if the assigned value was a constant, because the validation was previously implemented in a tricky way to work around the same design problem that prevented the validation rules from referring to anything other than the variable being validated.

(That itself was a compromise that allowed variable validation to ship even though Terraform's design didn't really allow for it yet; we've been improving the internal design in the meantime and that's what's finally allowed v1.9 to behave the way we originally intended.)

While I was implementing this more general version for v1.9 I became aware of that historical mistake, and so for backward-compatibility I implemented a workaround where Terraform blends the old special case with the normal expression evaluation rules, which means that var.test_input (the variable being validated) is known here even though it shouldn't really be under Terraform's normal rules. That special case does not apply to anything else, including other input variables.

If we were designing this over again from scratch then I think the ideal design would not perform variable validation during the validation phase at all (despite the nomenclature making that seem strange), because the validation phase is supposed to be checking whether each module in isolation is statically valid, and then the planning phase is the one responsible for deciding if dynamic expressions and values flowing between modules are valid. Alas, this historical quirk must be preserved under the 1.x compatibility promises.

I think all of that is separate from the root cause of this design gap anyway, though: even if this input variable were treated as known, we'd still need to decide how to treat unknown values in the error message to deal with all of the other sources of unknown values. If we solve it in the way I described earlier then the treatment of input variables during the validation phase is a moot point because we'd be evaluating your error message in the plan phase anyway, and so the variable would be known by then.

[apparentlymart]

Oh, and yes, you are right that I intended to say that the condition is known but the error_message is not, indeed.

[burnerlee]

hey @apparentlymart, I am looking forward to contribute to terraform. Not sure if this is a suitable issue for a beginner like me. Should I take this up or can you please assign me some other suitable issue?

[crw]

Hi @burnerlee, please also see the response to your earlier inquiry: https://github.com/hashicorp/terraform/pull/35408#issuecomment-2204515266

[KyleKotowick]

@apparentlymart In this case, I worked around this issue by doing the following:

variable "test_input" {
  type = bool
  validation {
    condition     = var.error_message != null && var.test_input == true
    error_message = var.error_message
  }
}

By using the error message variable in the condition, it forces the condition evaluation to wait until the error message value is known (plan phase). Since the error message variable is nullable = false, var.error_message != null will always return true so it won't effect the logic of the validation.

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.