terragrunt icon indicating copy to clipboard operation
terragrunt copied to clipboard

Can not set variable to null in Terragrunt inputs

Open yorinasub17 opened this issue 4 years ago • 12 comments

This needs investigation, but we were seeing some behavior where if you try to use null as one of the variable values in inputs, terragrunt translates it as the string "null" by the time it gets to terraform.

yorinasub17 avatar Oct 03 '19 23:10 yorinasub17

Did a quick test, @yorinasub17. If the variable is defined as a number then it will fail with a Error: Invalid value for input variable and explicitly with The environment variable TF_VAR_vm_default_capacity does not contain a valid value for variable "vm_default_capacity": a number is required. if you use null in the inputs.

If the variable is defined as a string, then it will parse null as a literal string instead: resource_group_name = "bastion-development-rg" -> "null-development-rg"

ashemedai avatar Nov 07 '19 16:11 ashemedai

This may be a limitation of terraform. I think if you run terraform apply -var foo=null, it parses null as a string.

brikis98 avatar Nov 07 '19 16:11 brikis98

I just ran into this same issue where I have some conditional code around null string variable handling to create a resource. Passing null from terragrunt.hcl negates the null handling, though when omitting the variable from inputs in terragrunt.hcl, null handling works as expected.

kerr-bighealth avatar Feb 20 '20 15:02 kerr-bighealth

I have faced this problem today as well.

ahmad-hamade avatar Aug 26 '20 22:08 ahmad-hamade

Hello all, I'm facing same issue which is blocking my terraform modules executions when using Terragrunt:

inputs = {
  optional = try(dependency.foo.outputs.optional, null)
}

output:

Error: Invalid value for input variable

The environment variable TF_VAR_optional does not contain a valid value for
variable "optional": a number is required.

I've just submitted a fix in the pull request 1367 (with option for old behavior)

masterleros avatar Sep 23 '20 02:09 masterleros

Same problem , trying to pass null to input for certain environments

faelnpaiva avatar Oct 07 '20 20:10 faelnpaiva

Same problem , trying to pass null to input for certain environments

You can use this approach to sort out the null approach avoiding to pass null values to the inputs (if you are using default values inputs in your modules, else it would fail)

inputs = {
  for key, val in {
    # ... your inputs ...
  } :
  key => val if val != null
}

Thanks @yorinasub17 for this tip!

masterleros avatar Oct 08 '20 01:10 masterleros

Looks like I faced the same issue:

include "envcommon" {
  path           = "${dirname(find_in_parent_folders())}/_envcommon/prod/rds/alpha/terragrunt.hcl"
  merge_strategy = "deep"
}

dependency "rds-alpha-master" {
  config_path = "${get_terragrunt_dir()}/../../../../eu-north-1/prod/rds/alpha"
}

inputs = {
  cross_region_replica = true
  replicate_source_db  = dependency.rds-alpha-master.outputs.db_instance_arn

  # Username and password should not be set for replicas
  username = null
  password = null

  multi_az = false

In rendered json I can see:

   + username                              = "null"

And then I get:

 Cannot change master user password on an RDS postgres Read Replica because it uses physical replication and therefore cannot differ from its parent.

suppix avatar Dec 21 '21 17:12 suppix

The problem has been relevant for three years, it seems they are not going to solve it, have there been any crutches or workarounds?

  • In the comments I saw so far only sort out the null approach

introquest avatar Jul 03 '22 16:07 introquest

The problem has been relevant for three years, it seems they are not going to solve it, have there been any crutches or workarounds?

Try generating a tfvars file that sets the variable to null?

lorengordon avatar Jul 03 '22 16:07 lorengordon

I have just hit this problem with a few variables that my module has defaults for. But they all turn out to be set to the string "null" in the state. Is there no traction on this after 3 years? Or is there another workaround like a conditional to leave the variable out from the inputs?

anitakrueger avatar Jul 13 '22 08:07 anitakrueger

I noticed that it got "null" when a variable is only defined as type = string , number or bool and put its value in terragrunt inputs = {}.

I hope that it will be fixed asap so I don't need to make a validation or condition in very variable

MaoChhaya avatar Jul 15 '22 04:07 MaoChhaya

Just had this issue too, what are people using as a workaround?

jonasthehobbit avatar Nov 29 '22 14:11 jonasthehobbit

wow, got this issue today. A big limitation.

DanielViglione avatar Dec 02 '22 10:12 DanielViglione

Also faced this problem.

ragumix avatar Jan 20 '23 12:01 ragumix

We're running into this problem as well.

Edit: Creating a terraform.tfvars file and setting the value to null there worked for me.

mwarkentin avatar Feb 01 '23 21:02 mwarkentin

@denis256 @yorinasub17 Randomly pinging a maintainers in hopes that this gets attention... Sorry... This issue is problematic.

pedrohdz avatar Feb 16 '23 12:02 pedrohdz

Just faced this issue today

AnhQKatalon avatar Mar 06 '23 18:03 AnhQKatalon

This is indeed rather ugly and limiting. I would like to not have to have a specific check for "null" strings.

ralucado avatar Mar 08 '23 10:03 ralucado

Faced this today, big limitation in Terragrunt IMO.

sherifabdlnaby avatar Mar 20 '23 12:03 sherifabdlnaby

I ran into this today.

This is implicit type coercion, and it would help to produce at least a warning message until passing nulls to inputs is supported.

rinerb avatar Mar 27 '23 16:03 rinerb

Hello everyone, quick update on this issue. This is indeed a limitation of terraform. The variable loader for environment variables and -var command line options intentionally does not evaluate expressions like the tfvars file loader does. We had previously prototyped a solution that uses tfvars files, but that opens up security risks that are unacceptable for some of our customers. I'll be keeping an eye on the relevant terraform issue and will do what I can to get a solution pushed through.

Until then, there are a few workarounds mentioned in this issue. Alternately, if you have control of the module you are trying to use the terraform team suggested the following workaround:

Set the type constraint for the variable to type = any, which tells Terraform to expect a value of any type, and then use a custom validation rule to restore the constraint that it must be something stringable:

variable "memory" {
  type = any

  validation {
    condition     = can(tostring(var.memory))
    error_message = "The \"memory\" value must be a string."
  }
}

You'd then be able to set the environment variable either to a literal string expression or to null:

TF_VAR_memory='"foo"' # single quotes to escape the literal double quotes
TF_VAR_memory='null'

Terragrunt directly translates your inputs into TF_VAR_ statements. The examples above would translate to:

inputs = {
  memory = "\"foo\""
}
inputs = {
  memory = null
}

ellisonc avatar Apr 04 '23 15:04 ellisonc

Hello @denis256, thank you for the update. Just fetched the latest version of Terragrunt for our project and saw that terragrunt init now always generates an empty .terragrunt-null-vars.auto.tfvars.json file in the repository even though we are only using hcl configs for our projects without empty inputs. Is there an option to deactivate the generation of the file with the help of a cli-flag or configuration similar to the skip_region_validation when generating the provider block?

dmatkovic avatar Aug 16 '23 17:08 dmatkovic

Hello, which issues cause having empty .terragrunt-null-vars.auto.tfvars.json ?

denis256 avatar Aug 16 '23 17:08 denis256

Hello, which issues cause having empty .terragrunt-null-vars.auto.tfvars.json ?

that's the question, right? I started having that file show up with just {} in it, too. I don't know why. I'm going to just add it to .gitignore

grimm26 avatar Aug 16 '23 18:08 grimm26

Hello @denis256, no immediate issue for the execution of the code but it would be nice if the generation could be skipped as described in #2670 so that not every terragrunt project in the future has to generate or ignore this file in the repository when there is no need for it in case of being empty.

dmatkovic avatar Aug 17 '23 07:08 dmatkovic

Hello everyone, quick update on this issue. This is indeed a limitation of terraform. The variable loader for environment variables and -var command line options intentionally does not evaluate expressions like the tfvars file loader does. We had previously prototyped a solution that uses tfvars files, but that opens up security risks that are unacceptable for some of our customers. I'll be keeping an eye on the relevant terraform issue and will do what I can to get a solution pushed through.

Until then, there are a few workarounds mentioned in this issue. Alternately, if you have control of the module you are trying to use the terraform team suggested the following workaround:

Set the type constraint for the variable to type = any, which tells Terraform to expect a value of any type, and then use a custom validation rule to restore the constraint that it must be something stringable:

variable "memory" {
  type = any

  validation {
    condition     = can(tostring(var.memory))
    error_message = "The \"memory\" value must be a string."
  }
}

You'd then be able to set the environment variable either to a literal string expression or to null:

TF_VAR_memory='"foo"' # single quotes to escape the literal double quotes
TF_VAR_memory='null'

Terragrunt directly translates your inputs into TF_VAR_ statements. The examples above would translate to:

inputs = {
  memory = "\"foo\""
}
inputs = {
  memory = null
}

I am confused with this workaround...

Isn't it the case that it is not possible to use single quotes in terragrunt to define strings?

Isn't it the case that environment variables set externally of terragrunt, are not picked up by terraform?

celsocoutinho-tangany avatar Feb 19 '24 12:02 celsocoutinho-tangany