read_terragrunt_config not evaluating dependency blocks

[Rwwn]

Describe the bug

With two Terragrunt files:

• file-A, which is the file being run
• file-B, which is being read by file-A using read_terragrunt_config

Running file-A results in errors being thrown about the dependencies in file-B not existing, e.g.

ERRO[0084] Error: Unsupported attribute
       
ERRO[0084]   on /Users/haynesr/repos/sdfe-terraformcontrol/terraform/terragrunt/infrastructure/team-infra/platform/gocd/test-server/infra/terragrunt.hcl line 53: 
ERRO[0084]   53:   hosted_zone_id      = dependency.env-global.outputs.public_domain_zone_id 
ERRO[0084]                                              
ERRO[0084] This object does not have an attribute named "env-global".

These dependency blocks are present in file-B, and work fine when running file-B directly, but do not work when running from file-A, unless you also add these same dependency blocks to file-A

This used to work fine on Terragrunt 0.53.1, but has happened since we upgraded to 0.58.13. The documentation for the read_terragrunt_config function also explicitly states that it should render dependency blocks from the file being read. However this does not seem to be the case.

Steps To Reproduce

Relevant code from file-A (config/terragrunt.hcl)

locals {
  infra_hcl  = read_terragrunt_config("../infra/terragrunt.hcl")
}

Relevant code from file-B (infra/terragrunt.hcl):

dependency "env-global" {
  config_path = "${get_parent_terragrunt_dir()}/infrastructure/env-bootstrap/global"
}

inputs = {
  hosted_zone_id      = dependency.env-global.outputs.public_domain_zone_id
}

This code will result in the error pasted above, unless you duplicate the env-global dependency from file-B into file-A.

Expected behavior

The read_terragrunt_config function should evaluate dependency blocks, so that the file being read from can use values from its dependencies, without them needing to be defined in the file doing the reading as well.

Nice to haves

• [ ] Terminal output
• [ ] Screenshots

Versions

• Terragrunt version: 0.58.13
• OpenTofu/Terraform version: 1.8.4
• Environment details (Ubuntu 20.04, Windows 10, etc.): MacBook M1

[yhakbar]

Thanks for reporting this, @Rwwn . We'll look into it!

[chuegle]

I believe this was broken in 0.54.16. The dependency is processed and put into a context, but that context is not used in later processing in the same file.

[brucehajdu]

Confirmed what @chuegle said, downgrading to 0.54.15 solves the problem.

EDIT: This appears to be exacerbated by an unrelated problem in tgenv only grabbing the 100 latest release tags. I can address this separately, but the problem referenced above is still valid.

[denis256]

Hello, I suspect something is missing in the steps to make this issue happen, I configured a similar setup and it is not failing on 0.54.16

.
├── app
│   ├── main.tf
│   ├── terraform.tfstate
│   └── terragrunt.hcl
├── global
│   ├── main.tf
│   ├── terraform.tfstate
│   └── terragrunt.hcl
├── infra
│   ├── main.tf
│   ├── terraform.tfstate
│   └── terragrunt.hcl
└── README.md

# app/terragrunt.hcl
locals {
  infra_hcl  = read_terragrunt_config("../infra/terragrunt.hcl")
}

inputs = {
  infra_hcl = local.infra_hcl
}

# infra/terragrunt.hcl
dependency "env-global" {
  config_path = "${get_parent_terragrunt_dir()}/../global"
}

inputs = {
  hosted_zone_id      = dependency.env-global.outputs.public_domain_zone_id
}

$ cd app
$ terragrunt.0.54.16 apply
...
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
...

It will be helpful to have failing example to debug why this happens

Full example in: https://github.com/denis256/terragrunt-tests/tree/master/issue-3310

[chuegle]

For my case, there's an include in there:

# test/infra/terragrunt.hcl
include "root" {
  path = find_in_parent_folders()
}

# test/infra/main.tf
variable "test" {
  type = number
}
output "test" {
  value = var.test
}

# test/terragrunt.hcl
locals {
  app = read_terragrunt_config("${get_path_to_repo_root()}/test/app")
}

inputs = {
  test = local.app.inputs.test
}

# test/app/terragrunt.hcl
dependency "global" {
  config_path = "${get_path_to_repo_root()}/test/global"
  mock_outputs = {
    test = 1
  }
}

inputs = {
  test = dependency.global.outputs.test
}

# test/global/terragrunt.hcl
locals {}

# test/global/main.tf
output "test" {
   value = 1
}

with terragrunt version 0.54.15:

> (cd test/infra && terragrunt plan)
Initializing the backend...
...
Changes to Outputs:
  + test = 1
....

with terragrunt version 0.54.16:

> (cd test/infra && terragrunt plan)
ERRO[0006] Error: Unsupported attribute

ERRO[0006]   on /pathstuff/test/app/terragrunt.hcl line 9:
ERRO[0006]    9:   test = dependency.global.outputs.test
ERRO[0006]
ERRO[0006] This object does not have an attribute named "global".
...

[chuegle]

@denis256 have you got the chance to investigate it yet? I know before that dependency objects weren't available to locals but were available in input, but now they aren't available to input, which basically makes them unusable in an include after the changes in 0.54.16.

[applevladko]

same issue here probably related to https://github.com/gruntwork-io/terragrunt/issues/3036 (which is marked as resolved)

[mwarkentin]

We are experiencing this issue as well, in all releases after 0.54.15.

Our units import our base terragrunt config:

include "base" {
  path = find_in_parent_folders()
}

That file reads the terragrunt config of the whole hierarchy of our tree like this:

root_config    = read_terragrunt_config(find_in_parent_folders("root.hcl", "${get_parent_terragrunt_dir()}/empty.hcl"))
  regions_config = read_terragrunt_config(find_in_parent_folders("regions.hcl", "${get_parent_terragrunt_dir()}/empty.hcl"))
  tenancy_config = read_terragrunt_config(find_in_parent_folders("tenancy.hcl", "${get_parent_terragrunt_dir()}/empty.hcl"))
  silo_config    = read_terragrunt_config(find_in_parent_folders("${basename(get_terragrunt_dir())}.hcl", "${get_parent_terragrunt_dir()}/empty.hcl"))
  # group config is an optional config that is used for groups of services like kafka, clickhouse, postgres, pops, redis, buckets, etc.
  group_config   = read_terragrunt_config(find_in_parent_folders("group.hcl", "${get_parent_terragrunt_dir()}/empty.hcl"))
  service_config = read_terragrunt_config(find_in_parent_folders("service.hcl", "${get_parent_terragrunt_dir()}/empty.hcl"))
  local_config   = read_terragrunt_config("${get_terragrunt_dir()}/local.hcl", { inputs = {}, locals = {} })

And any dependency {} blocks defined in those layers is now triggering the This object does not have an attribute named "...". error on all versions 0.54.16 and above.

[chaosun-abnormalsecurity]

Hi @denis256 @yhakbar just wanna check on the update on this. It seems still happening and impacting users who use this pattern of dependency configurations. Thanks

[yhakbar]

Hey folks, sorry this hasn't been resolved for so long.

TBH, I don't remember the context for this issue, but on reflection here, I don't think users should be reading dependencies when using read_terragrunt_config. Dependencies are very expensive to resolve, as they require running OpenTofu/Terraform, and it seems like a good thing that we don't do that when running read_terragrunt_config for performance reasons (though it seems to me that this is a bug, and not an intentional optimization).

Can someone please clarify what their business use-case is for using read_terragrunt_config on Terragrunt configurations with a dependency, and reading that dependency value out? I'd like to get a better understanding of the value of this.

[chaosun-abnormalsecurity]

Hey folks, sorry this hasn't been resolved for so long.

TBH, I don't remember the context for this issue, but on reflection here, I don't think users should be reading dependencies when using read_terragrunt_config. Dependencies are very expensive to resolve, as they require running OpenTofu/Terraform, and it seems like a good thing that we don't do that when running read_terragrunt_config for performance reasons (though it seems to me that this is a bug, and not an intentional optimization).

Can someone please clarify what their business use-case is for using read_terragrunt_config on Terragrunt configurations with a dependency, and reading that dependency value out? I'd like to get a better understanding of the value of this.

Thanks @yhakbar. This comment can show our use case.

[yhakbar]

@chaosun-abnormalsecurity

I see the configuration, but I'm trying to understand the business value. If you want to read values from another unit, why not read outputs from it using a dependency block? I don't see the value in using read_terragrunt_config and would like to understand that better.

[mwarkentin]

@yhakbar for us it's a reduction of duplication within our units. Our unit/terragrunt.hcl files generally look like this:

include "base" {
  path = find_in_parent_folders("base.hcl")
}

terraform {
  source = "[email protected]:getsentry/terraform-modules//cluster_clickhouse?ref=c15ced676f1bd62c6fa65f924b1c9fa12a36514d"
}

Within that included base.hcl, we walk back up the tree with read_terragrunt_config() and build up our set of locals / inputs for the module. So now as soon as we reference a dependency from one of these levels on the tree, it errors in the newer versions of terragrunt.

Edit: I should also point out that it works well in older versions, and the performance of calling out to terraform hasn't been an issue for us. Generally we act on individual units, not a lot of large terragrunt run-all commands, so maybe that is part of why we haven't noticed a performance problem.

Edit 2: This seems similar to the terragrunt example here: https://github.com/gruntwork-io/terragrunt-infrastructure-live-example/blob/main/prod/us-east-1/prod/mysql/terragrunt.hcl#L9-L11

However we don't include the common configurations from the unit, we do it from our root.hcl: https://github.com/gruntwork-io/terragrunt-infrastructure-live-example/blob/main/prod/us-east-1/prod/mysql/terragrunt.hcl#L15-L19

[chuegle]

@yhakbar Our use case is very similar to above, where it's just an attempt to keep things DRY across different instances of thing.

root.hcl
definitions/{widget_type}.hcl
instances/{location}/{widget_owner}/{widget_name}/{widget_type}/terragrunt.hcl

terragrunt.hcl includes root.hcl, then root.hcl determines that it needs to load the widget_type.hcl definition. That leads to read_terragrunt_config("definitions/widget_type.hcl"). That widget_type.hcl can define that it uses a dependency on some other widget type for that location/owner.

read_terragrunt_config - It seems reasonable that part of the config you would read would be values of dependencies.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for raising this issue.

[mwarkentin]

Not sure if this will reopen it, but still waiting response.