terragrunt

terragrunt copied to clipboard

I am having a project structure as follows: . ├── README.md ├── modules │ ├── function-app-consumption │ │ ├── locals.tf │ │ ├── main.tf │ │ ├── output.tf │ │ └── variables.tf │ └── storage-account │ ├── main.tf │ ├── output.tf │ └── variables.tf └── sandbox ├── eastus │ ├── regional.tfvars │ ├── functionapp-svc │ │ └── terragrunt.hcl │ └── storageaccount-function │ └── terragrunt.hcl ├── environment.tfvars └── terragrunt.hcl

I want to deploy the infrastructure in "sandbox" environment. the root terragrunt.hcl file for that env. looks like this:

generate "provider" {
  path      = "_provider.tf"
  if_exists = "overwrite"
  contents  = <<EOF

provider "azurerm" {
  features {

  }
}

EOF
}

remote_state {
  backend = "azurerm"

  disable_init = tobool(get_env("TERRAGRUNT_DISABLE_INIT", "false"))

  generate = {
    path      = "_backend.tf"
    if_exists = "overwrite"
  }

  config = {
    resource_group_name  = "my-sandbox-rg"
    storage_account_name = "mysbxtfbackend"
    container_name       = "tfstate"
    key                  = "${path_relative_to_include()}/terraform.tfstate"
  }
}

I am using the TERRAGRUNT_DISABLE_INIT environment variable to disable backend initialization while running the validate-all command. like this:

cd sanbox/
export TERRAGRUNT_DISABLE_INIT=true
terragrunt validate-all

It works fine when there is no dependency defined in any of the modules. But when a module has a dependency like this:

terraform {
  source = "../../../modules//function-app-consumption"

  extra_arguments "custom_vars" {
    commands = get_terraform_commands_that_need_vars()

    arguments = [
      "-var-file=${get_terragrunt_dir()}/../regional.tfvars",
      "-var-file=${get_terragrunt_dir()}/../../environment.tfvars"
    ]
  }

}

dependency "storage_account" {
  config_path                             = "../storageaccount-function"
  mock_outputs_allowed_terraform_commands = ["validate", "plan"]
  mock_outputs = {
    storage_account_name       = "mockstgacc"
    storage_primary_access_key = "mockaccesskey"
  }
}

dependencies {
  paths = ["../storageaccount-function"]
}

include {
  path = find_in_parent_folders()
}


inputs = {

  function_app_name          = "func-my-sbx-svc"
  function_version           = "~3"
  reserved                   = true
  os_type                    = "linux"
  storage_account_name       = dependency.storage_account.outputs.storage_account_name
  storage_account_access_key = dependency.storage_account.outputs.storage_primary_access_key

  app_settings = {
    "FUNCTIONS_WORKER_RUNTIME" = "java",
    "WEBSITE_RUN_FROM_PACKAGE" = "1"
  }
}

It starts failing , complaining that it requires backend initialization.

[terragrunt] [/Users/abhi/codes/infra/sandbox/eastus/functionapp-svc] 2021/02/17 16:28:38 Running module /Users/abhi/codes/infra/sandbox/eastus/functionapp-svc now
[terragrunt] [/Users/abhi/codes/infra/sandbox/eastus/functionapp-svc] 2021/02/17 16:28:38 Running command: terraform --version
[terragrunt] [/Users/abhi/codes/infra/sandbox/eastus/functionapp-svc] 2021/02/17 16:28:38 Terraform version: 0.14.4
[terragrunt] [/Users/abhi/codes/infra/sandbox/eastus/functionapp-svc] 2021/02/17 16:28:38 Reading Terragrunt config file at /Users/abhi/codes/infra/sandbox/eastus/functionapp-svc/terragrunt.hcl
[terragrunt] [/Users/abhi/codes/infra/sandbox/eastus/storageaccount-function] 2021/02/17 16:28:38 Generated file /Users/abhi/codes/infra/sandbox/eastus/storageaccount-function/.terragrunt-cache/644837049/_backend.tf.
[terragrunt] [/Users/abhi/codes/infra/sandbox/eastus/storageaccount-function] 2021/02/17 16:28:38 Running command: terraform init -get=false -get-plugins=false
[terragrunt] [/Users/abhi/codes/infra/sandbox/eastus/storageaccount-function] 2021/02/17 16:28:40 Running command: terraform output -json

Error: Initialization required. Please see the error message above.

As we can see from the logs, the command terraform init -backend=false is not run , instead, the command terraform init -get=false -get-plugins=false gets executed. This seems to be a bug.

Ah yes that does look like a bug, specifically, it should skip the direct remote state based dependency fetching when disable_init is true. We're a bit buried at the moment to implement this, but if anyone wants to submit a PR, the function in question is here: https://github.com/gruntwork-io/terragrunt/blob/579258163dd05878f2b3cfa53611c0f5fb8a728e/config/dependency.go#L427

As far as workarounds go, does it work if you add disable_dependency_optimization = tobool(get_env("TERRAGRUNT_DISABLE_INIT", "false")) to the remote_state block?

No @yorinasub17 , I tried adding disable_dependency_optimization = tobool(get_env("TERRAGRUNT_DISABLE_INIT", "false")) to the remote_state block , but it was of no help.

Ah upon further inspection, I see that we are handling errors from the output command and special casing when there is no output to return the mocks. That is sort of expected behavior given that the dependency handlers are assuming a clean output call, and output depends on being able to read the state properly (e.g. we don't want to silently fail and return mocks for auth issues).

Does it work if you have skip_outputs = tobool(get_env("TERRAGRUNT_DISABLE_INIT", "false")) on the dependency block?

Yep it worked!! Thanks a lot for the workaround :-) @yorinasub17

can something like this be fixed please?

I have fixed this issue by just adding init command in mock_outputs_allowed_terraform_commands:

dependency "network" {
  config_path = "../network"

  mock_outputs_allowed_terraform_commands = ["validate", "init"]
  mock_outputs = {
    private_subnets_ids = [
        "subnet-xxxxxxxxxxxxxxxxx"
    ]
    nat_public_ips      = [
        "xx.xxx.xxx.xxx"
    ]
  }
}

dependencies {
  paths = ["../network"]
}

inputs = {
  private_subnets_ids = dependency.network.outputs.private_subnets_ids
  nat_public_ips      = dependency.network.outputs.nat_public_ips
}

Running into this issue too :(

Happy to pick up on this if not assigned, btw is the preferred flow picking from the To Do pile?