scaffolding icon indicating copy to clipboard operation
scaffolding copied to clipboard

Duplicate global address in terraform modules

Open cmurphy opened this issue 9 months ago • 0 comments

Description

The redis and mysql modules each manage a google_compute_global_address resource with name <cluster name>-priv-ip: https://github.com/sigstore/scaffolding/blob/0621d5066ef1019615d922bf2e886f97fb93b7aa/terraform/gcp/modules/mysql/mysql.tf#L40 https://github.com/sigstore/scaffolding/blob/0621d5066ef1019615d922bf2e886f97fb93b7aa/terraform/gcp/modules/redis/main.tf#L35

If the modules are being applied as part of the umbrella sigstore module, then cluster_name will be the same for each and they will try to create the same resource named the same thing, and terraform will abort with a conflict error:

Error: Error creating GlobalAddress: googleapi: Error 409: The resource 'projects/<my project>/global/addresses/<my cluster name>-priv-ip' already exists, alreadyExists

The only way around this is to import the "existing" resource (which terraform just created for the other module) and run terraform apply again.

If no infrastructure had been deployed yet and we were working from a blank slate, an easy fix would be to move the address resource out of the redis and mysql submodules into the sigstore umbrella module: https://gist.github.com/cmurphy/e65913dd4533d2733a1c3404851b103e

This is not safe to do in an already-deployed environment, because terraform sees the resource as new and destroys the old ones, assigning a new IP range to the new resource.

Terraform provides a moved block to do this type of refactoring, so resources aren't destroyed. However, this won't work to move two resources, because it's ambiguous to terraform:

│ Error: Ambiguous move statements
│ 
│   on .terraform/modules/sigstore/terraform/gcp/modules/sigstore/sigstore.tf line 169:
│  169: moved {
│ 
│ A statement at .terraform/modules/sigstore/terraform/gcp/modules/sigstore/sigstore.tf:164,1 declared that module.sigstore.module.mysql.google_compute_global_address.private_ip_address moved to module.sigstore.google_compute_global_address.private_ip_address, but this statement instead declares that
│ module.sigstore.module.redis.google_compute_global_address.service_range moved there.
│ 
│ Each resource can have moved from only one source resource.

The only way to resolve this on a running system would be to

  1. Add the move statement for one of the resources, say mysql
  2. Manually remove the other resource from the terraform state with terraform state rm.
  3. Run terraform apply

This is not a friendly migration for anyone consuming the terraform modules from this repo.

Version

6d618ccb23ee773e578cd65fbaff7f5ac24865d3

cmurphy avatar May 10 '24 21:05 cmurphy