-refresh=false doesn't work with terraform "cloud"

[tmccombs]

Terraform Version

Terraform v1.6.2
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v5.22.0

Your version of Terraform is out of date! The latest version
is 1.6.3. You can update by downloading from https://www.terraform.io/downloads.html

Terraform Configuration Files

terraform {
  cloud {
    organization = "my-organization"
    hostname = "tfe.example.com" # terraform enterprise endpoint
    workspaces {
      name = "my-workspace"
    }
  }
}

Debug Output

N/A

Expected Behavior

Supplying the -refresh=false should disable refreshing even if using terraform cloud or terraform enterprise.

Actual Behavior

As far as I can tell supplying -refresh=false on the command line does nothing if using terraform enterprise or terraform cloud with a terraform.cloud configuration. At least if VCS is set up for the workspace.

I still see output saying that the state for all resources is being refreshed.

Steps to Reproduce

1. Set up terraform with terraform cloud or enterprise
2. Run a plan or apply with the -refresh=false flag
3. Watch as that flag has no effect

Additional Context

The reason that I want to use -refresh=false is that when I am developing new configuration I sometimes want to run speculative plans to see the actual changes my code will produce. Needing to wait for the state to be refreshed for all resources slows that down, which makes iteration slower. I'd like to speed up the process so that I can iterate more quickly. when actually making a pull request, doing a full refresh is fine.

As a sidenote, it is also annoying that I can't use the TF_LOG environment variable to get more verbose logs, or set an environment variable for a single speculative run. If I need to get debug logs, I have to to the settings of the workspace, set the TF_LOG environment variable, do the run, then go back and set it back, which is rather annoying.

References

No response

[apparentlymart]

Thanks for reporting this, @tmccombs.

It does seem that there's code in the Cloud integration to pass that option on to the Terraform Cloud API:

https://github.com/hashicorp/terraform/blob/10f4567fcc284cc649d939705a8b2a70c339b951/internal/cloud/backend_plan.go#L219-L225

The Refresh argument here is the documented way to disable refreshing when creating a run with the Terraform Cloud API, so at first glance it seems like the code in this repository is upholding its part of the contract.

I can also see in the Terraform Cloud Agent codebase (whose source code isn't public) some logic that adds the -refresh=false option to the command it's running when Terraform Cloud commands the agent job to "skip refresh".

Therefore it isn't clear to me exactly what's missing here, but there's a lot of other Terraform Cloud control plane code between the two "ends" I am familiar with and so I think we'll need to await someone with more familiarity with Terraform Cloud to understand exactly what's going on here.

---

If you have been able to capture some TF_LOG=trace output using the technique you described, it might be helpful to see the "CLI args:" log line from the early part of that output, which should look something like this:

2023-11-06T17:54:20.256-0500 [INFO]  CLI args: []string{"terraform", "plan"}

In your case I'd hope to see at least one more argument "-refresh=false". If that's not present then it would confirm that this flag is getting "lost" somewhere in the chain between your local Terraform CLI and the remote one running in Terraform Cloud Agent.