terragrunt Terragrunt 0.50.15 is 15x slower when your Terragrunt repo has multiple dependencies in my terragrunt.hcl files.

Describe the bug Terragrunt release v0.50.15 is much slower at calculating multiple dependencies. The commands terragrunt graph-dependencies, terragrunt run-all plan and terragrunt run-all apply are all x15 slower in my scenario.

In releases before v0.50.15, terragrunt graph-dependencies takes ~0.5 seconds to execute. With release v0.50.15+ , the same command command takes ~14 seconds to execute.

To Reproduce I put together a example repo to reproduce this issue: https://github.com/Simon-TheUser/terragrunt-performance-changes

Run terragrunt graph-dependencies from the /iac folder in the project..

Sample hcl:

include "root" {
  path   = find_in_parent_folders("root.hcl")
  expose = true
}

dependencies {
  paths = [
    find_in_parent_folders("vm_setup_post-cloud-init"),
    find_in_parent_folders("vm_setup_ssh")
    ]
}

dependency "vm" {
  config_path = find_in_parent_folders("vm_instance/terragrunt.hcl")
}

inputs = {
  ip_address  = dependency.vm.outputs.vm_ip
  vm_uuid     = dependency.vm.outputs.vm_uuid
}

Expected behavior A performance losses are sometimes necessary, but I would not expect a x15 performance drop.

Nice to have Here is the time it takes to run

$ ./03-print-results.sh
Testing terragrunt graph-dependencies version 0.43.3
real    0m 0.66s
user    0m 0.85s
sys     0m 0.09s

Testing terragrunt graph-dependencies version 0.50.13
real    0m 0.58s
user    0m 0.75s
sys     0m 0.08s

Testing terragrunt graph-dependencies version 0.50.14
real    0m 0.58s
user    0m 0.79s
sys     0m 0.05s

Testing terragrunt graph-dependencies version 0.50.15
real    0m 14.88s
user    0m 20.01s
sys     0m 2.21s

Testing terragrunt graph-dependencies version 0.50.16
real    0m 14.58s
user    0m 19.89s
sys     0m 1.99s

Testing terragrunt graph-dependencies version 0.51.7
real    0m 14.23s
user    0m 19.17s
sys     0m 2.19s

Testing terragrunt graph-dependencies version 0.53.6
real    0m 11.46s
user    0m 11.23s
sys     0m 1.20s

Versions

Terragrunt version: 0.50.15
Terraform version: 1.3.10 *not important.
Environment details (Ubuntu 20.04, Windows 10, etc.): Running inside a Podman container. Base image is alpine:3.18.3

Nov 24 '23 00:11 Simon-TheUser

It's the first time I work with Go code. I'm not exerienced.

In configstack/module.go, I modified resolveDependenciesForModule and resolveModules to pass to each other the variable moduleMap. This seems to resolve the performance / recursion problem but the test case TestResolveTerraformModulesMultipleModulesWithNestedExternalDependencies fails.

I will create a PR if I can get my changes to pass the unit test.

Nov 27 '23 20:11 Simon-TheUser

I' ve tried to upgrade to 0.55.1 from 0.45.8 and found that i also have this issue in project with multiple dependencies and complicated dependency graph. Was forced to downgrade to 0.50.14. Thx for benchmark

Feb 12 '24 15:02 Vetrenik

Hi, it's same for me too. I had to rollback to 0.50.17 as the latest (including 0.55.11) is too slow and takes more than 5 minutes for single module (terragrunt apply) with 2 dependencies.

Mar 03 '24 17:03 logamanig

Hi, will be helpful to get feedback on changes from https://github.com/gruntwork-io/terragrunt/pull/2983 to see if performance is improved for real users