terragrunt conflicts with tfenv

[MacFlurry]

Hello,

OS macOS 10.14 trying to install terragrunt

$ brew install terragrunt

Updating Homebrew... ==> Auto-updated Homebrew! Updated 1 tap (homebrew/core). No changes to formulae.

==> Installing dependencies for terragrunt: terraform ==> Installing terragrunt dependency: terraform Error: Cannot install terraform because conflicting formulae are installed. tfenv: because tfenv symlinks terraform binaries

Please brew unlink tfenv before continuing.

Unlinking removes a formula's symlinks from /usr/local. You can link the formula again after the install finishes. You can --force this install, but the build may fail or cause obscure side-effects in the resulting software.

So terragrunt won't install because of Tfenv exists. Any help ?

[brikis98]

The Homebrew installer for Terragrunt is maintained by the community, so to be honest, I'm not sure what is the right fix for this. Suggestions and PRs are very welcome!

[nitrocode]

@MacFlurry I ran into this same issue and this was my workaround. I use tfenv to use the terraform version 0.11.7

Verify before

$ terraform --version
0.11.7

Workaround will unlink tfenv, install terragrunt which installs the latest version of terraform (which it depends on), and then we can relink tfenv to use the original 0.11.7 version

brew unlink tfenv && brew install terragrunt && brew link tfenv

Verify after

$ terraform --version
0.11.7

And according to terragrunt --help, it looks like it will default to use the terraform on PATH.

GLOBAL OPTIONS:
   ...
   terragrunt-tfpath                    Path to the Terraform binary. Default is terraform (on PATH).

[udondan]

You can just skip dependencies when installing:

brew install --ignore-dependencies terragrunt

Or in a Brewfile (Homebrew Bundle):

brew "awscli"
brew "aws-okta"
brew "terraform_landscape"
brew "terragrunt", args: ["ignore-dependencies"]
brew "tfenv"

Then it can be installed via brew bundle.

But I'd also prefer terraform not to be a hard dependency. Though I couldn't find the source for the Homebrew config to propose a change.

[evanstoddard23]

An issue with using a Brewfile with ignore-dependencies is that flag is ignored on upgrades, giving the same linking issues. Since homebrew core doesn't support options which would allow terraform install to be optional, I'm considering a PR to remove terraform as a dependency and adding info about installing terraform/tfenv in the caveats section. Curious what others, especially maintainers might think about this.

[blairham]

The real fix is to just remove the dependency, as terragrunt does not require it to actually execute, it's an external dependency. Anyone using terragrunt, already knows they will need terraform, as it will also give you an error if you run it without terraform installed.

[mmindenhall]

Can we take another crack at getting this done?

https://github.com/Homebrew/homebrew-core/pull/73326 was closed because the reviewer noted that it's a poor user experience to install terragrunt via homebrew and have it not work due to terraform not being installed. I don't disagree, but installing and updating terraform with homebrew does not work well for teams managing a lot of infrastructure across multiple providers with terraform! For example:

• We have had team members unintentionally update remote state to a new terraform version after doing a brew upgrade that included a newer version of terraform. This causes the terraform apply for other users (and CI/CD pipelines!) to fail until they are also upgraded, because terraform won't let you run apply with an older version than what was captured in remote state. To prevent this from recurring, we added the required_version attribute to all of our terraform stacks.
• When our required_version is older than the latest available in homebrew, it's hard for team members to stay in sync with homebrew.
  • Users doing a fresh install can't just brew install terraform. Instead, they need to find an earlier homebrew commit where the required version was the lastest, check out that commit, then run brew install terraform.
  • All users need to disable homebrew "auto cleanup" so the required version doesn't get deleted every time a new version is installed with brew upgrade
  • After every brew upgrade, the team needs to run brew switch terraform <older version> to re-activate the correct version.

Also, the experience for tfenv users is not good. Similar to above, I now have to do the following every time there's a terraform or terragrunt homebrew update:

brew unlink tfenv; brew link terraform; brew upgrade; brew unlink terraform; brew link tfenv

I think terraform should absolutely be removed as a dependency, but a caveats section can be added that will be displayed to users after running brew install terragrunt (or brew info terragrunt) that instructs users to install terrafrom either via brew install tfenv; tfenv install <terraform version> (preferred) or brew install terraform.

[mmindenhall]

I'm going to submit a new PR that removes terraform as a dependency, and adds the following caveats section. Please chime in with any additions or edits you think might be helpful!

  def caveats
    <<~EOS
      Terragrunt has been installed as
        #{HOMEBREW_PREFIX}/bin/terragrunt

      Terragrunt requires a version of `terraform` to be in the user's path.

      Teams using terragrunt/terraform need to use the same version of
      terraform. To prevent accidental version changes via `brew upgrade`, we
      recommend using `tfenv` to install and manage `terraform` versions:
        brew install tfenv
        tfenv install <required terraform version>
        tfenv use <required terraform version>

      See: https://github.com/tfutils/tfenv

      Terraform can also be installed directly with homebrew.  This is only
      recommended when NOT working as part of a team (i.e., for solo projects,
      learning, or experimenting):
        brew install terraform
    EOS
  end

[mmindenhall]

No joy! In the first PR it sounded like homebrew was amenable to the removal of terraform as a dependency as long as there were clear instructions to the user. In the 2nd PR, they made it clear that even with clear instructions for how to install terraform separately, they will not allow removal of terraform as a dependency in homebrew-core, because then the formula may not work when installed. They suggested creating a separate homebrew tap, then the formula can be managed however we want. If someone at gruntwork wants to take this up:

1. Create a new tap repo under the gruntwork-io account. If there are other gruntwork tools that might be included in the future, it would probably make sense to name the repo homebrew-gruntwork instead of homebrew-terragrunt.
2. Add the terragrunt formula (similar to what was done in https://github.com/Homebrew/homebrew-core/pull/75836)
3. Create a PR against homebrew-core to remove the existing formula
4. Update the docs with instructions to install using the new tap.

[mmindenhall]

I was able to implement a workaround for this using a git post-merge hook. Here's how to do it:

1. cd $(brew --repo homebrew/core) to get to the repo root.

2. From there, create a file under .git/hooks called post-merge. The file should contain the following (if you already have a post-merge file for some reason, add the code after the first line):

   #!/usr/bin/env bash
   
   changed_files="$(git diff-tree -r --name-only --no-commit-id ORIG_HEAD HEAD)"
   
   check_run() {
   	echo "$changed_files" | grep --quiet "$1" && eval "$2"
   }
   
   # Remove terraform dependency from terragrunt
   check_run terragrunt.rb "sed -i '' 's/depends_on \"terraform\"//g' Formula/terragrunt.rb"
   

3. Make sure the file is executable: chmod +x .git/hooks/post-merge

Note that if you use gnu-sed (installed as gsed) instead of sed native in macOS, you'll need to modify the last line slightly:

check_run terragrunt.rb "gsed -i 's/depends_on \"terraform\"//g' Formula/terragrunt.rb"

And if you are running linux, I think this should work:

check_run terragrunt.rb "sed -i 's/depends_on \"terraform\"//g' Formula/terragrunt.rb"

With this hook, whenever you run brew update and terragrunt.rb is included in the update, the dependency on terraform will be removed from the updated file. Then when you run brew upgrade, the upgrade will succeed without having terraform installed via homebrew.

I was worried that having a modified version of the terragrunt formula would cause an error the next time brew update ran with a terraform change, but there was just an update today that succeeded, so I think this works.

[jav-12]

So at this point what is the best way to avoid the terraform dependency in Terragrunt. This is very annoying every time Terragrunt needs to be updated and the solution given above didn't work for me.

[lorengordon]

Based on the response from the Homebrew folks, sounds like just don't use brew to install terragrunt.

So at this point what is the best way to avoid the terraform dependency in Terragrunt. This is very annoying every time Terragrunt needs to be updated and the solution given above didn't work for me.

[mmindenhall]

@jav-12, what's your dev environment (mac, windows, linux)?

[jav-12]

Ups sorry. I'm using Mac BigSur @mmindenhall

[mmindenhall]

Hi, this approach is working for me and I assume (from the 🚀 responses) at least a few others. My first suggestion is to double-check that you did everything above correctly:

1. You created the post-merge file in the .git folder within the homebrew-core folder (mine is under /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core.
2. Ensure that the post-merge file has execute permissions.
3. Make sure that you're using the version of sed that comes pre-installed in MacOS (running the command which sed should return /usr/bin/sed). If you have installed another version of sed (e.g., gnu sed from homebrew) you'll need to change that last line to make the intended change with your version of sed.

If that still doesn't work, you could write a small script that will manually run the sed command against the terragrunt.rb file. Run the command manually before the next time you run brew upgrade. Then run the command after any brew upgrade that includes a new version of terragrunt. You could also create an alias to do this automatically after every upgrade so you don't have to think about it:

alias brewup='brew upgrade && ~/scripts/fix-terragrunt.sh'

Hope that helps!

[jorhett]

I've made a PR trying the options used here https://stackoverflow.com/questions/21115259/alternative-dependencies-in-a-homebrew-formula-e-g-for-use-with-gcc

If they won't accept this, perhaps we should add a tap to this repo?

[blairham]

I'm very sorry to necro this. A co-worker brought me back to this. I've resolved this using asdf instead of Homebrew to install Terragrunt and Terraform.