Flattening terraform.tfvars

[Dyllaann]

Hi! I am getting quite close to being migrated to pretf, however I think I am in front of the last challenge to solve. Jinjaform used to be able to flatten the directory structure, keeping intact all variables set in terraform.tfvars by (i believe) merging them together. Currently, only the closest terraform.tfvars is being symlinked into the current directory. To make things a little more transparent in which files are being used, I took the Jinjaform approach and made a subfolder where everything will be pulled into.

from pretf import workflow
from pathlib import Path
import os

def pretf_workflow(path):
    if not Path('pretf').is_dir():
        Path('pretf').mkdir()
    os.chdir('pretf/')

    # Restrict where pretf/terraform can run to directories containing an auto
    # tfvars file. It will show an error when running in the wrong directory.
    workflow.require_files("*.tfvars")

    # Flatten the directory structure into the working directory.
    #workflow.delete_links()
    created = workflow.link_files("*.tfvars", "*.tf", "*.tf.py", "*.tfvars.py", "modules")

    # Now run the standard Pretf workflow which generates files
    # and then executes Terraform. Pass in the mirrored files
    # so they can be cleaned up.
    return workflow.default(clean=False)

For example:

Service/
    prod/
        terraform.tfvars
    qa/
        terraform.tfvars
    vm.tf
    variables.tf
    terraform.tfvars

Whenever I run pretf init inside Service/prod, only the Service/prod/terraform.tfvars will end up in Service/prod/pretf, leaving the Service/terraform.tfvars behind, thus missing variable values.

[raymondbutcher]

Hi. Yes Jinjaform created new files and merged the contents from files in parent directories. It doesn’t work here with symlinks, as you’ve seen, but you can make it work.

1. That is clever moving into a pretf subdirectory before it runs, I never thought of that. It might mess with require_files thinking it can run from that temp directory though. The main reason I moved away from temp directories was dealing with state files. Some state commands like pull and push get confusing if they end up downloading into a different directory. But feel free to do this if you want, that’s the whole point of allowing custom workflows!

2. Rename the Service directory’s terraform.tfvars to service.auto.tfvars - it should get symlinked in with its different name and still loaded because Terraform loads all *.auto.tfvars files along with terraform.tfvars.

3. Change the require_files line to look for terraform.tfvars so it will prevent you from accidentally running in the Service directory.