terragrunt icon indicating copy to clipboard operation
terragrunt copied to clipboard

Terragrunt run-all evaluates locals of not included modules when --terragrunt-strict-include is used

Open MichalKoziorowski-TomTom opened this issue 1 year ago • 7 comments

Terragrunt run-all evaluates locals of not included modules when --terragrunt-strict-include is used

For reference: https://github.com/gruntwork-io/terragrunt/blob/v0.54.12/configstack/module.go#L59 https://github.com/denis256/terragrunt-tests/tree/master/issue-2860

denis256 avatar Jan 02 '24 20:01 denis256

Any news on this? It breaks the entire deployment as in our case staging has no key to production sops file and vice versa

egorksv avatar Mar 11 '24 20:03 egorksv

Hi. Are there any news with PR for this bug? I have really ugly workarounds in my TG modules because of this (filling not needed env variables with fake values in unrelated modules).

https://gist.github.com/egorksv/65ab1a1ab8bc5cfdb7b3a6330e90b5af

Workaround script to force include specific modules

group/*.hcl are empty hcl files that are used to logically group modules

egorksv avatar Apr 10 '24 13:04 egorksv

I just started to see "new" version of this bug.

  • If not included module evaluates locals, deployment fails.
  • If not included module evaluates inputs, I see errors but deployment continues.

In effect, I need to to provide to terragrunt env variables of modules which I don't want to deploy.

I wanted only my "datacollector" application to be redeployed, and I've got errors about cluster that was not expected to be touched:

['terragrunt', 'run-all', 'apply', '--terragrunt-working-dir=/infrastructure/kubernetes/terragrunt', '--terragrunt-strict-include', '--terragrunt-include-module-prefix', '--terragrunt-log-level=info', '--terragrunt-non-interactive', '--terragrunt-include-dir=applications/datacollector']
time=2024-04-10T13:44:17Z level=info msg=The stack at /infrastructure/kubernetes/terragrunt will be processed in the following order for command apply:
Group 1
- Module /infrastructure/kubernetes/terragrunt/applications/datacollector


time=2024-04-10T13:44:23Z level=error msg=Error: Error in function call

 prefix=[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
time=2024-04-10T13:45:21Z level=error msg=  on /infrastructure/kubernetes/terragrunt/cluster/azure_tags/terragrunt.hcl line 16:
 prefix=[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
time=2024-04-10T13:45:21Z level=error msg=  16:   deployer            = "${get_env("DEPLOYER")}"
 prefix=[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
time=2024-04-10T13:45:21Z level=error msg=
 prefix=[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
time=2024-04-10T13:45:21Z level=error msg=Call to function "get_env" failed: EnvVarNotFoundError: Required environment
variable DEPLOYER - not found.

 prefix=[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
time=2024-04-10T13:45:21Z level=warning msg=Error reading partial config for dependency azure_tags: /infrastructure/kubernetes/terragrunt/cluster/azure_tags/terragrunt.hcl:16,28-36: Error in function call; Call to function "get_env" failed: EnvVarNotFoundError: Required environment variable DEPLOYER - not found. prefix=[/infrastructure/kubernetes/terragrunt/cluster/dns_zone] 
time=2024-04-10T13:45:59Z level=info msg=Downloading Terraform configurations from file:///infrastructure/kubernetes/terraform into /infrastructure/kubernetes/terragrunt/applications/datacollector/.terragrunt-cache/PrxYmuk-qgovdK4RDn3zHprxfsU/MuvRh0xLYIb1VzakUyVr62McSpg prefix=[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
[/infrastructure/kubernetes/terragrunt/applications/datacollector] Initializing the backend...
[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
[/infrastructure/kubernetes/terragrunt/applications/datacollector] Successfully configured the backend "azurerm"! Terraform will automatically
[/infrastructure/kubernetes/terragrunt/applications/datacollector] use this backend unless the backend configuration changes.
[/infrastructure/kubernetes/terragrunt/applications/datacollector] Initializing modules...
[/infrastructure/kubernetes/terragrunt/applications/datacollector] - azure_tags in ../../modules/azure_tags
[/infrastructure/kubernetes/terragrunt/applications/datacollector] 

TG version: v0.55.19

Hi, regarding the PR https://github.com/gruntwork-io/terragrunt/pull/3012

Just a question, would it also address dependency evaluation of the terragrunt stack, and not just locals evaluation?

For my use case, I append a .skip to the path name to environments or stacks we intend to skip. e.g. rename uat/ to uat.skip/. I use --terragrunt-include-dir along with --terragrunt-strict-include to specify exactly which stacks to execute. My pipeline checks for this path and handles it automatically.

However, since Terragrunt currently does not fully respect the --terragrunt-include-dir and --terragrunt-strict-include, when another stack which uat.skip/ depends on gets executed (e.g. networkhub), it will fail because the stacks inside uat.skip/ are still trying to evaluate the original path uat/.

e.g. uat.skip/tgw depends on networkhub/tgw. So when running terragrunt run-all plan --terragrunt-strict-include --terragrunt-include-dir networkhub/tgw, it still evaluates uat.skip/tgw - in my case causing a failure as the stacks under uat.skip are looking for the original uat path.

It really should fully respect the strict flags and not even evaluate it in the first place

amirasyraf avatar Apr 22 '24 01:04 amirasyraf

Hi @denis256. I see that there is a draft PR attached to this issue. Do you maybe plan to have this finished?