Environment variables should be evaluated after yaml parse

[justinkillen]

When there is a variable in the yaml that isn't defined the stack fails as expected. If I disable that stack, it still fails. If I comment out the line, it still fails. It seems like the variable replacement should be moved until after the yaml parse, or maybe the variable replacement to be taught to ignore disabled and/or commented lines.

stacker-1.0.4

[ejholmes]

I've run into this problem a few times as well in the past. Although vars from the environment file look similar to lookups, evaluating happens more like rendering a template.

I think it'd probably make sense to deprecate the ${var_name} syntax, and make it a lookup like ${env var_name}, which would solve this problem I think.

Another way around this is to just not rely on stacker's environments, and use something to generate the stack config before you hand it off to stacker. For example:

#!/usr/bin/env python

import os
import yaml

vpc_stack = {
    'name': 'vpc',
    'class_path': 'blueprints.VPC'}

bastion_stack = {
    'name': 'bastion',
    'class_path': 'blueprints.Bastion'}

stacks = [
    vpc_stack]

if os.environ.get('BASTION_ENABLED'):
    stacks.extend([bastion_stack])


config = {
    'stacks': stacks}

print yaml.dump(stacks)

$ ./gen.py | stacker build -

The benefit of this is it gives you a ton of flexibility in how you build the config file before letting stacker execute everything.

[jesse-peters]

I run into this issue regularly when developing new stuff. It would be great if we parsed it earlier - I think the idea of defining it as an env lookup makes a lot of sense.

Do lookups get processed before the env file? If so it would be easy enough just to create a new lookup type. That might what you were suggesting above :)

[ejholmes]

@syphon7 lookups get processed at runtime, as needed, whereas env files are processed like a template as one of the first steps of a stacker build/destroy.

The stacker.yaml file gets interpolated as a strings.Template using the env file as data. Then this "rendered" template is what is passed down throughout stacker:

https://github.com/remind101/stacker/blob/9d51cc34274356b374336def3c467b4814365aa1/stacker/config/init.py#L12-L39

[ejholmes]

Just realized that an ${env ...} lookup wouldn't actually work for this, since those are only available inside variables, and the env file interpolation is pretty commonly used outside variables.