Inconsistent consecutive evaluation inside required_var_files section

[MichalKoziorowski-TomTom]

Describe the bug Multiple calls of terragrunt command produces different terraform commands.

I'm having list of locations in which I have var files with variables. I want them to be applied in a specific order so I can make variable value overrides. Not all of var files must exist, so I'm making checks and array concatenation inside required_var_files section.

terraform {
  # Use double slack ("//") in the path so that terragrunt can support the terraform module
  # to reference its sibling modules using a relative path (e.g. ../modules/azure_tags).
  # Reference: https://developer.hashicorp.com/terraform/language/modules/sources#modules-in-package-sub-directories
  source = "${get_terragrunt_dir()}/../../../terraform//applications/worker"

  extra_arguments "conditional_vars" {
    commands = [
      "apply",
      "destroy",
      "plan"
    ]

    # https://terragrunt.gruntwork.io/docs/features/keep-your-cli-flags-dry/#required-and-optional-var-files
    # Syntax below is little hacky, but we want to have var files used in specific order, and the first file might not exist.
    required_var_files = concat(
      fileexists("${get_working_dir()}/config/region/${local.common_env_vars.inputs.cluster_region}.tfvars") ? ["${get_working_dir()}/config/region/${local.common_env_vars.inputs.cluster_region}.tfvars"] : [],
      [
        local.config_settings_file_path,
        local.config_versions_file_path
      ]
    )
  }
}

This module is part of a more complex deployment, which is deployed using terragrunt run-all apply ... in root directory. The problem is that even though my file always exists, it sometimes appears in terraform apply command produced by terragrunt, and sometimes not. I've enabled debug option and see following produced terraform commands:

First call:

DEBU[0705] Running command: terraform apply 
-var-file=/home/koziorow/dev/routing-api-config/values/applications/common/dev/settings.tfvars 
-var-file=/home/koziorow/dev/routing-api-config/values/applications/api/worker/dev/versions.tfvars 
-auto-approve -input=false  prefix=[/home/koziorow/dev/routing-api-infrastructure/kubernetes/test1/applications/worker]

One of the next calls:

DEBU[0186] Running command: terraform apply 
-var-file=/home/koziorow/dev/routing-api-infrastructure/kubernetes/test1/applications/worker/.terragrunt-cache/k-OJprzEIcpZQMOyCZjyv-n9vKM/ToQ4raEpNckryvJojEUWnt7bG8E/applications/worker/config/region/westeurope.tfvars 
-var-file=/home/koziorow/dev/routing-api-config/values/applications/common/dev/settings.tfvars 
-var-file=/home/koziorow/dev/routing-api-config/values/applications/api/worker/dev/versions.tfvars -auto-approve -input=false  prefix=[/home/koziorow/dev/routing-api-infrastructure/kubernetes/test1/applications/worker]

File that is sometimes added and sometimes not is located in terraform module I'm trying to deploy, in config/region/westeurope.tfvars location.

Expected behavior Consecutive terragrunt executions should produce same terraform command

Nice to have

• [ ] Terminal output
• [ ] Screenshots

Versions

• Terragrunt version: v0.53.8
• Terraform version: v1.4.1
• Environment details (Ubuntu 20.04, Windows 10, etc.): Ubuntu 22.04

[MichalKoziorowski-TomTom]

It must be some threading issue or related to the order in which operations are performed in terragrunt.

If I replace the reference to working dir:

fileexists("${get_working_dir()}/config/region/${local.common_env_vars.inputs.cluster_region}.tfvars") ? ["${get_working_dir()}/config/region/${local.common_env_vars.inputs.cluster_region}.tfvars"] : [],

with just a path to config inside the terraform module:

fileexists("${get_terragrunt_dir()}/../../../terraform/applications/worker/config/region/${local.common_env_vars.inputs.cluster_region}.tfvars") ? ["${get_terragrunt_dir()}/../../../terraform/applications/worker/config/region/${local.common_env_vars.inputs.cluster_region}.tfvars"] : [],

then everything works. Otherwise, I need to apply two or three times.

[denis256]

Hello, I tried to reproduce issue in https://github.com/denis256/terragrunt-tests/tree/master/issue-2834

but I get same order of env var files:

DEBU[0000] Running command: terraform apply -var-file=/projects/gruntwork/terragrunt-tests/issue-2834/app2/.terragrunt-cache/yGqT3pr1MdNAZNBAC8k5-7p0Ylo/9QlQVibzYpXJB6lQ0M6kQ2UqmkA/us-east-1.tfvars -var-file=settings.tfvars -var-file=versions.tfvars -var-file=1.tfvars -var-file=2.tfvars -var-file=3.tfvars -auto-approve -input=false  prefix=[/projects/gruntwork/terragrunt-tests/issue-2834/app2] 

DEBU[0000] Running command: terraform apply -var-file=/projects/gruntwork/terragrunt-tests/issue-2834/app3/.terragrunt-cache/bZ0BH7tORY15OFNW9ZNEmYBH4aQ/V76B35WRaSTg7zFltX68X4Z0rt0/us-east-1.tfvars -var-file=settings.tfvars -var-file=versions.tfvars -var-file=1.tfvars -var-file=2.tfvars -var-file=3.tfvars -auto-approve -input=false  prefix=[/projects/gruntwork/terragrunt-tests/issue-2834/app3] 

can be shared repo there the issue happens?