Expected way to share runtime variables between pre-api and terraform steps

[CalvinRodo]

Since version 1.5.0 of AFT the bash scripts and the terraform share the same container and as is stated:

This allows sharing runtime variables or configurations between these stages. Both per-account and global customization workflows now use this pattern.

What is the expected way to share runtime variables between the pre-api and the terraform step?

[CalvinRodo]

This isn't a bug I'm just not sure how to enter in questions

[hanafya]

Hey @CalvinRodo!

One way you can accomplish is by using Variable Definitions (.tfvars) Files. Please see, https://developer.hashicorp.com/terraform/language/values/variables#variable-definitions-tfvars-files

In addition, you can modify the CLI Configuration here https://developer.hashicorp.com/terraform/cli/config/config-file

[hsdp-smulford]

Maybe set the variables you want to persist in terraform as TF_VAR_...: https://developer.hashicorp.com/terraform/cli/config/environment-variables#tf_var_name

[CalvinRodo]

@hsdp-smulford How would you recommend I do that? The pre-api-helpers.sh script while running in the same container as the terraform apply is by the very nature of how bash scripts work run in a different process from the terraform apply, so there really isn't a way to set an environment variable in this pipeline.

@hanafya Yeah I'm aware of tfvars files in fact that's how I've actually solved this problem, however it's not ideal as I basically have to write an .auto.tfvars file outside of the script running directory, so for global it needs to be created in $DEFAULT_PATH/terraform/ and for account customizations it's created here $DEFAULT_PATH/$CUSTOMIZATION/terraform/.

So while that works it is undocumented and I had to read the codepipeline scripts to find out how the folder structures are setup. Since it's not explicit in documentation I don't think there is any real requirement that the developers of this project keep it like that so I would like to know the official guidance on this problem so I don't have to go changing up my scripts if they refactor in the future.

[awsomeinfra]

@CalvinRodo I am facing as issue where aft account customisation pipeline is not able to read .tfvars or .auto.tfvars file written into aft-account-customisation-repo terraform configuration. Its seems you have solved this. y creating .auto.tfvars file in $DEFAULT_PATH/$CUSTOMIZATION/terraform/. Could you please share how you have achieved this.

[CalvinRodo]

So I didn't end up using an auto.tfvars file, what I ended up doing was using the custom-fields field in the aft-account-request modules link to docs which creates a parameter under /aft/account-request/custom-fields/ in the vended account.

Then we just use that parameter in the data source of the customization type we wanted in this case the aft-global-customizations repo, but it will work for account-customizations as well.

We did this using the following example code for a custom-fields named foo

data "aws_ssm_parameters_by_path" "custom" {
  path = "/aft/account-request/custom-fields"
}

locals {
  custom_field_found = contains(data.aws_ssm_parameters_by_path.custom.names, "/aft/account-request/custom-fields/foo") ? 1 : 0
}

data "aws_ssm_parameter" "foo" {
  count = local.custom_field_found
  name  = "/aft/account-request/custom-fields/custom_field"
}

Then you just reference the custom value like would for any resource behind a feature flag so

data.aws_ssm_parameter.foo[0]

[CalvinRodo]

if you want to use a bash script we basically just added the following to the api-helps/pre-api-helpers.sh script.

FOO_VAL=$(aws ssm get-parameter --name /aft/account-request/custom-fields/foo --region ca-central-1 --output json --query Parameter.Value)
echo "FOO_VAL: $FOO_VAL"

if [[ $FOO_VAL == "null" ]]; then
    echo "No foo found"
    # Set to empty so the variable exists when running terraform apply
    FOO_VAL=""
else

FOO_VAR_FILE="$DEFAULT_PATH/terraform/foo.auto.tfvars"
# Because environment variables can't be exposed out of a bash script
cat << EOF > "$FOO_VAR_FILE"
foo_val="$(tr -d '"' <<< "$FOO_VAL")"
EOF

fi

cat "$FOO_VAR_FILE"

Then referenced it in the code using

variable "foo_val" {
  type = string
}

I'm pretty sure it worked and we just dropped it because a pure terraform solution was much nicer.