Terraform 1.4.0/1.4.1: terraform init fails with "Backend configuration changed"

[hameno]

Terraform Version

Terraform v1.4.1
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v4.58.0
+ provider registry.terraform.io/hashicorp/azuread v2.36.0

Terraform Configuration Files

...terraform config...

Debug Output

Initializing the backend...
2023-03-16T09:30:38.713+0100 [TRACE] Meta.Backend: built configuration for "http" backend with hash value 4239287872
2023-03-16T09:30:38.714+0100 [TRACE] Preserving existing state lineage "60bbbff8-5ad3-fa1c-a614-fa68e6012e9c"
2023-03-16T09:30:38.715+0100 [TRACE] Preserving existing state lineage "60bbbff8-5ad3-fa1c-a614-fa68e6012e9c"
2023-03-16T09:30:38.715+0100 [TRACE] Meta.Backend: working directory was previously initialized for "http" backend
2023-03-16T09:30:38.715+0100 [TRACE] backendConfigNeedsMigration: configuration values have changed, so migration is required
2023-03-16T09:30:38.715+0100 [TRACE] Meta.Backend: backend configuration has changed (from type "http" to type "http")

Expected Behavior

terraform init works

Actual Behavior

terraform init does not work

Steps to Reproduce

1. terraform init with < 1.4.0
2. terraform init with >= 1.4.0

Additional Context

http backend is used, after running -reconfigure the terraform.tfstate file shows that

            "client_ca_certificate_pem": null,
            "client_certificate_pem": null,
            "client_private_key_pem": null,

was added.

References

No response

[jbardin]

Hi @hameno,

Thanks for filing the issue. Do you know what the exact prior condition of the .terraform/terraform.tfstate file was, and what version it was originally created with? Initializing a backend and removing any null attributes does not appear to cause the same behavior you are reporting.

Thanks!

[hameno]

State was like this:

{
    "version": 3,
    "serial": 1,
    "lineage": "7c767314-e0af-a690-de1d-16cfdf92ed76",
    "backend": {
        "type": "http",
        "config": {
            "address": "https://gitlab.example.com/api/v4/projects/1/terraform/state/test",
            "lock_address": "https://gitlab.example.com/api/v4/projects/1/terraform/state/test/lock",
            "lock_method": "POST",
            "password": "TOKEN",
            "retry_max": null,
            "retry_wait_max": null,
            "retry_wait_min": 5,
            "skip_cert_verification": null,
            "unlock_address": "https://gitlab.example.com/api/v4/projects/1/terraform/state/test/lock",
            "unlock_method": "DELETE",
            "update_method": null,
            "username": "USER"
        },
        "hash": 1687282793
    },
    "modules": [
        {
            "path": [
                "root"
            ],
            "outputs": {},
            "resources": {},
            "depends_on": []
        }
    ]
}

It happens for every terraform project created prior to 1.4.0

[jbardin]

Thanks @hameno, it seems this is caused by a change in the http backend schema imposed by #31699. If any of the backend parameters are supplied via cli flags, the check for a changed configuration is being triggered. We can look into trying to prevent the need to reconfigure the backend.

[ianburrowes]

Like @hameno I am also using a GitLab-managed Terraform state and am affected by this issue, after upgrading to Terraform v1.4.+

Running a terraform plan on an existing project tells me that the backend configuration has changed, and I am not able to reinitialise it.

Reverting to v1.3.9 solves the issue.