hashicorp
Copy deferred data sources into the plan, and use deferred values in references/outputs
This PR has two authors and two goals — @liamcervante was already working on wiring deferred values into outputs, and I needed to build on some of what he'd already done in order to finish implementing deferral for data sources.
Data sources can cause deferred changes
Previously, we weren't copying any information about deferred data sources into the plan; the theory was that data sources are handled during plan, and any resource that refers to them will already be listed as deferred, so we can just reconstruct things during the apply as needed.
This turned out to be not quite coherent, mostly because data sources CAN in fact produce planned changes that get processed during an apply! To observe this, just make a data source reference a computed attribute from a resource that hasn't been applied yet; the data source will plan a Read action. This caused at least one bug with the prior approach, when a data source was both deferred due to a deferred prereq AND delayed due to unknown configuration -- it would plan a change that would blow up during apply.
So, since deferral is conceptually nearly the same thing as a delayed read (it just gets delayed til a future plan/apply cycle instead of the upcoming apply), we're now making deferred data sources produce a deferred Read change in the plan. This makes the storage formats for data sources and resources in the Deferred struct identical, which simplifies several things. It also lets us easily produce deferred graph nodes for data sources in the apply, which simplifies getting their incomplete values for use in outputs.
Outputs use deferred change values
When building the hcl eval context for evaluating expressions, we now use the deferred change planned value for resources and data sources if one is available. This generally seems to result in outputs getting appropriately null values for things that can't be known due to deferral.
Questions for reviewers
- Is the new behavior of outputs coherent, in the case where a resource previously existed in the state but we've got a deferred planned change coming up? They're now returning the (partial or unknown) planned value, rather than the old fully-known (but soon-to-be invalidated) value. I went in circles on this.
- In the new logic for partial-expanding data sources, do we need to do anything to handle data sources that are nested in check blocks? I didn't think so, but I'm also new to check blocks.
- For @liamcervante: Could you expand a bit on the changes to transform_destroy_edge and node_resource_apply? This was part of the outputs stuff, and it's the one bit I didn't fully understand. I think it's ultimately because outputs might be referring to partially-expanded resources inside modules that have unknown for_each, but I couldn't fully connect the dots yet.
Target Release
1.9.x
Draft CHANGELOG entry
EXPERIMENTS
The deferred actions experiment now handles data sources and outputs more thoroughly.
