Feature request: semver constraints for module references.

[jgiles]

It would be extremely useful if we could express terraform module references with semver constraints:

terragrunt = {
  terraform {
    source = "[email protected]:example/example-terraform.git//modules/mymodule"
    version = "^1.0.1"
  }
}

This would save users from widespread source code updates just to pick up patch fixes.

Terraform added support for semver constraints in module source versions in https://github.com/hashicorp/terraform/issues/15613, but only for registry-sourced modules. https://github.com/hashicorp/terraform/issues/17994 is the request for git-sourced modules.

terragrunt could wait for the terraform feature to land, but would not necessarily have to. Updating the terragrunt-pinned versions for patch changes in terraform code is currently a pretty big pain point for us; the version updates tend to drown out any other changes.

The syntax above also suggests an interesting possibility: The module could actually be specified in a higher-level terraform.tfvars file, meaning that in the common case of a parallel terraform + terragrunt repo, you could specify a "default" version of the terraform repo once. That approach wouldn't play especially well with referencing multiple repos, though.

[brikis98]

Terragrunt currently uses terraform init -from-module under the hood to download code, so whatever that supports, we should support. To add support for SemVer directly in Terragrunt, we would need to be able to get a list of all available tags in a repo. That means either (a) checking out the repo manually or (b) using APIs from GitHub, BitBucket, GitLab, etc. Given the number of version control systems, and different auth systems, (b) seems impractical. (a) might be doable, at least for Git, but then we'd need different codepaths for Terraform Registry, Git, and everything else...

I also worry that magically bumping patch numbers is a risk. It means every time you run apply, you're potentially deploying something different. It makes it harder to promote a single, immutable version from environment to environment, with tests at each stage. I guess if you only make the "patch" version auto-update, it should in theory be 100% backwards compatible, but I'd still argue it makes it harder to reason about the code. Before, every change would be captured (a) in the code and (b) in the commit log. With version constraints, figuring out what is actually deployed requires knowing when apply was executed (which isn't recorded in the VCS or possibly anywhere) and correlating those timestamps against releases of all module version numbers... Not a fun exercise when debugging a prod issue.

[lorengordon]

It might help to implement something upstream, either in terraform (a new argument init -from-module <...> -module-version <...>) or in go-getter (a query arg ?version=).

[picardsrcd]

has this feature request gained legs over time? I would love to allow pinning for patch level ranges. the ?ref= does not allow for ranges :(

(we are using github as the module source with tags for versions using semantic-release (see approach https://itnext.io/automating-tagging-and-versioning-of-terraform-modules-or-any-language-3a271966c63c)

[sflanker]

FWIW, at least when using a Terraform Registry as a source, terragrunt uses a custom go-getter, not terraform init -from-module: https://github.com/gruntwork-io/terragrunt/blob/master/internal/tfr/getter.go#L94

It is true that the code paths would differ slightly between Git and Terraform Registry for how to list versions, but I don't think it would be necessary, or a good idea, to rely on host specific APIs, just list tags for the git remote. The code for matching a version from the list based on the constraint string would then be the same regardless of the source. And once the version was selected the rest of the code would be pretty much the same as it is now.

And conveniently the semver constraint matching logic is readily available github.com/hashicorp/go-version so no need to reinvent that wheel.

I think it goes without saying that a version constraint would be unsupported and raise an error for a file path source.