Feature request: support latest version when using `tfr` protocol in `terraform.source`

[korenyoni]

trafficstars

Describe the solution you'd like

Use case: when using the tfr protocol in terraform.source, do not pin the module version in every single terragrunt.hcl configuration; only pin module versions in production configurations.

Imagine you have 50 terragrunt.hcl configurations and a the terraform.source attribute needs to be bumped across all of them following a module release.

Even using Renovate, Dependabot, or custom workflows triggered by module releases via a webhook (for example via Spacelift's notification policy) seems like unnecessary automation for non-production environments.

Right now, the version URI query parameter must be specified when supplying a URI with the tfr:// protocol.

terraform {
  source = "tfr:///terraform-aws-modules/vpc/aws?version=3.5.0"
}

Otherwise, an error is raised:

https://github.com/gruntwork-io/terragrunt/blob/df9f8792c33920dece415c26edbd9e1415aca585/internal/tfr/getter.go#L105-L110

This differs from vanilla Terraform in that the version attribute is optional.

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
}

Granted, in vanilla Terraform, since version is its own HCL attribute, it neither has to exist nor be a pure semantic version. It can be a version constraint such as ~> 3.0. Given version being optional and and also the possibility to supply version constraints, I can only assume that it has an additional phase where it makes use of the version query endpoint, then uses that constraint (or lack thereof) to determine the latest permissible version and then download it via this endpoint. Interestingly, I couldn't get the get the latest version of a module endpoint to work with all private registries, namely the Spacelift module registry.

I tried finding the actual implementation of this in https://github.com/hashicorp/terraform but I think it'll take me less time to make a PR to this repo than to find it...

In any case, I propose lifting the hard requirement of a version parameter in the tfr/getter implementation in Terragrunt. If version is missing, tfr/getter can make use of the version query endpoint to determine the latest version of the module.

Describe alternatives you've considered

There are a few options here:

1. what I am proposing — lift the hard version requirement in the tfr:// URI and make use of the version query endpoint if it is not supplied.
2. Do nothing — the tfr:// URI should exactly conform to the download module source endpoint.
3. Have the tfr:// URI should conform to the get the latest version of a module endpoint (but see my earlier notes of this endpoint not working for Spacelift, for example).
4. Make the version like the HCL attribute in vanilla Teraform — I wouldn't recommend this because the URI will probably not be able to support version constraints such as ~> 3.0, and also we would have to implement our own version constraint resolution mechanism inside of Terragrunt. Lastly, this would be misleading as a tfr:// URI because the Terraform registry API won't support version constraints inside the URI. All of this sounds awful.

Additional context

Refer to use case detailed at the top of this PR.

[scr-oath]

I'm also hitting this - I expected version=~> 1.2.3 to work (or escaping the > and space) but… it seems to get

    * error downloading 'tfr://...?version=~+0.0.20': Error downloading module from .../~%!.(MISSING)20/download: error receiving HTTP data

need some mechanism to specify version constraints

[cmlccie]

I propose adding a version attribute to the terraform block and fully implementing the Terraform Module Registry Protocol supporting version constraints. If the version attribute is null then, use the latest version provided by the Module Registry Protocol. This approach should provide parity between the Terragrunt and Terraform module source references and initialization.

These references would refer to the same modules:

Terraform:

module "registry_demo" {
    source = "myregistry.io/my-registry/my-module/aws"
    version = "~>1.20.0"
}

Terragrunt:

terraform {
    source = "tfr://myregistry.io/my-registry/my-module/aws"
    version = "~>1.20.0"
}

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for raising this issue.

[yhakbar]

Hey folks,

This issue is relevant to folks, but to a limited subset of the community as far as I can tell.

Maintainers have other irons in the fire that we're prioritizing instead of this. I've preserved the issue so that it won't go stale again, and marked it as requesting contributions if any member of the community is willing to tackle it.

We will review PRs open that tackle this.