`terraform state rm` does not respect local state backend's `workspace_dir` when creating state backup

[cspotcode]

Terraform Version

Terraform v1.5.1
on linux_amd64

Terraform Configuration Files

Debug Output

Expected Behavior

When using local state with custom workspace_dir, terraform state rm stores the state backup into the correct workspace_dir

Actual Behavior

terraform state rm tries and fails to save state backup into the default directory which does not exist.

2023-06-23T23:38:39.169Z [TRACE] statemgr.Filesystem: creating backup snapshot at terraform.tfstate.d/workspace-a/terraform.tfstate.1687563517.backup
Error saving the state: failed to create local state backup file: open terraform.tfstate.d/workspace-a/terraform.tfstate.1687563517.backup: no such file or directory

The state was not saved. No items were removed from the persisted
state. No backup was created since no modification occurred. Please
resolve the issue above and try again.

terraform.tfstate.d does not exist because I have set workspace_dir to .terraform-qa.tfstate.d, so that is where state files reside.

I can confirm this by looking at .terraform-qa/terraform.tfstate (I am also using a custom TF_DATA_DIR pointing to .terraform-qa)

{
    "version": 3,
    "serial": 1,
    "lineage": "336bd162-9070-27cc-079c-6884d4b3246f",
    "backend": {
        "type": "local",
        "config": {
            "path": null,
            "workspace_dir": "./terraform-qa.tfstate.d"
        },
        "hash": 666019178
    },
    "modules": [
        {
            "path": [
                "root"
            ],
            "outputs": {},
            "resources": {},
            "depends_on": []
        }
    ]
}

Steps to Reproduce

terraform init w/local state backend, custom backend config workspace_dir Attempt to terraform state rm something from the state.

Additional Context

No response

References

No response

[crw]

Thanks for this issue report! If you are viewing this issue and would like to indicate your interest, please use the 👍 reaction on the issue description to upvote this issue. We also welcome additional reports. Thanks again!

[svengreb]

I'm also facing this problem and I'd say this is not a "use case" but a bug. I export the TF_CLI_ARGS_init environment variable with -backend-config=/path/to/local-backend-options.hcl as value where the file contains the path and workspace_dir fields, both set to a custom location (maybe a unnecessary detail, but I do this because the custom path contains all my TF states which are auto-synced with my self-hosted Seafile instance in order to backup the states automatically). Anyway, using the terraform state rm command will not take this into account which is kind of expected due to the scope of the environment variable being the init command, but I still think this is a bug because Terraform should take this path into account. The required metadata is stored in the .terraform/terraform.tfstate file where the custom backend path is information of it (backend.config.workspace_dir field). I manage my states with workspaces, but I also checked that the directory within the backend.config.workspace_dir path contains the required directory (named like my workspace) and terraform workspace show also yields the correct name.

One workaround I applied is to also export the TF_CLI_ARGS_state_rm environment variable with -backup=/path/to/my/custom/workspace_dir/$(terraform workspace show)/terraform.tfstate.backup as value so that Terraform uses the file in the custom workspace directory instead of the default terraform.tfstate.d directory, located in the current working directory. The $(terraform workspace show) subshell inline call allows to dynamically use the correct directory of the currently active workspace.

I know that this setup might not be the one of a "common user", but setups like this often also bring up the edge cases where bugs are hidden 😉