Allow configuring default hostname for localterraform.com

[tmccombs]

Terraform Version

Terraform v1.7.2
on linux_amd64

Use Cases

Terraform allows you to use a generic hostname to reference the current instance of terraform cloud or terraform enterprise as the module registry.

However, when working on a module that doesn't have a remote backend configured, this domain doesn't work.

In my case, I have multiple instances of terraform enterprise, in different network partitions, that can't talk to each other. and want to be able to share module code between them, so I want to be able to use a single domain that references the current instance. But the current solution for that means I can't run terraform validate on my modules on workstations or in CI.

Attempted Solutions

The above linked documentation states:

To test configurations on a developer workstation without the remote backend configured, you must replace the generic hostname with a literal hostname in all module sources and then change them back before committing to VCS.

This is, unfortunately, extremely inconvenient, especially if there are a large number of modules involved.

Proposal

If an environment variable is set (maybe TF_REGISTRY_DOMAIN), then replace instances of localterraform.com in module sources with that domain if there is not a terraform remote configured.

References

No response

[crw]

Thanks for this feature request! If you are viewing this issue and would like to indicate your interest, please use the 👍 reaction on the issue description to upvote this issue. We also welcome additional use case descriptions. Thanks again!

[apparentlymart]

Hi @tmccombs,

Terraform does have an existing mechanism for overriding the service discovery process for a particular hostname, which is intended primarily for development but is also, I believe, how Terraform Enterprise is internally making localterraform.com work.

I've not tested this, but you might be able to get a suitable result by placing something like the following in your CLI configuration file:

host "localterraform.com" {
  services = {
    "modules.v1"   = "https://tfe.example.com/api/registry/v1/modules/"
    "providers.v1" = "https://tfe.example.com/api/registry/v1/providers/"
  }
}

What this block does is tell Terraform CLI that any time it would normally perform remote service discovery for localterraform.com it should use the hard-coded result in the services argument instead. Therefore I copied into here the most relevant entries from Terraform Cloud's service discovery document, but replaced app.terraform.io with tfe.example.com since I understand from your question that you are using Terraform Enterprise rather than Terraform Cloud.

As discussed in https://github.com/hashicorp/web-unified-docs/issues/790, there is some uncertainty right now about whether this mechanism is "officially supported" vs. just a development shim, but it sounds like you are interested in this primarily for development anyway, since presumably in your real Terraform Enterprise environments TFE is configuring this for you automatically anyway.

The main caveat with this approach is that it totally replaces the network service discovery for this hostname, and so if Terraform Enterprise changes the endpoint paths used for these services in future then you'll need to manually update your CLI configuration to match.

---

This localterraform.com hack seems to be getting a bit out of hand in terms of how much it's sprawling around all of the Terraform codebases and so it might be time to think a little harder about that problem to see if we can find a solution to it that doesn't require sprinkling weird little exceptions for that hostname in different spots.

Perhaps we could add an explicit way to tell Terraform CLI that it should treat one hostname as an alias for another -- not limited just to localterraform.com -- and then Terraform Enterprise could continue to use that domain as its convention but we'd also get the flexibility for any hostname someone might want to configure, if for example they need to migrate their private registry services from one hostname to another and so need a shim to support an old hostname as a locally-configured alias for a while during migration.

[tmccombs]

Using the host configuration seems to work. Although, I'm a little nervous about using an undocumented/unsupported feature like that.

I also figured out that in order for it to work with a token form configuration I had to copy the credentials config in ~/.terraform.d/credentials.tfrc.json for my original domain to a value for localterraform.com.

[apparentlymart]

Ahh yes, that's a good additional thing that ought to be supported if we choose to expose a "hostname aliases" feature more broadly.

Since we've started, I'll list a variety of things that we should probably ponder if we decide to expose an explicit way to configure aliases for hostnames:

• (As above) Should Terraform use the credentials from the alias target instead of those for the aliased hostname? Or should it support either and have some defined preference order for which to take?
• For hostnames appearing in module and provider addresses, should Terraform automatically rewrite the addresses to use the target hostname? Or vice-versa? For example, this might mean that the dependency lock file ends up tracking localterraform.com/foo/bar instead of example.com/foo/bar if localterraform.com is defined as an alias for example.com.
  • If the answer to the above is yes, should Terraform also accept addresses containing the target hostname but rewrite them to the alias? That is, effectively treating the alias "backwards" for the sake of constructing unique identifiers?
• Should terraform login accept an alias hostname for logging in? If it does, should it save the credentials it obtained as belonging to the alias, to the target, or both?

These questions all seem very answerable, but it'll require some deeper consideration of all of the use-cases associated with each of these applications of service hostnames than I can do immediately right now, so I'm mainly just leaving this as a note for me or someone else picking this up again in future.

Originally localterraform.com existed only as a special case in Terraform Enterprise, generating a host block like I previously described. It was later retrofitted into a small number of special cases in Terraform CLI to make it easier to develop modules intended for Terraform Enterprise. But it does seem like we're finding the limits of treating this as a hard-coded exception, and so I think it's time to design a general answer that can be explicitly configured in the CLI configuration and is then taken into account in a well-defined way by all of Terraform's features that use service hostnames. (Of course, the current special case of implicitly adding an alias for localterraform.com if you use a cloud block in your root module would need to remain, because that's protected by compatibility promises.)

---

FWIW, Since Terraform Enterprise is itself depending on being able to generate host blocks in the CLI Configuration, any attempt to remove that capability in future would need to have a replacement immediately available, and the old way would need to remain supported for some time to keep older installations of Terraform Enterprise working, and so as long as you're doing this only for local development of modules that would be used "for real" only in Terraform Enterprise I don't think there's much risk in depending on this workaround in the meantime. One day you might need to swap it out for some other setting, but there would be plenty of notice that it's coming.

[msteller]

First thank you for this solution. Works great when running locally for modules and providers using the generic hostname! We did run across an issue with a remote state data call to retrieve data. We updated the hostname and received an error when running a plan. "Error: Host localterraform.com does not provide a tfe service". So, we interrogated the service discovery (https://your-instance-url.com/.well-known/terraform.json) and found tfe.v2.X. We updated the service block in the terraform.rc file to include "tfe.v2.1" = "https://your-instance-url.com/api/v2/" and was able to successfully do a plan. Just wanted to share in case anyone else comes across this issue.