Support string substitutions in other places than Parameters

[alytle]

Being able to add variable substitution in other places than just Parameters which are imported would be valuable.

For example, I would love to be able to do something like this:

Variables:
    ProjectName: myproject

Stages:
  dev:
    DDBTable1:
      StackName: ${Variables.ProjectName}-dynamodb-${StageName}

And have my Stack Name come out as myproject-dynamodb-dev.

I'd be willing to work on this if you agreed it would be a useful feature.

[alytle]

One more example explaining why I'm thinking about this. I tend to have Stage names embedded in my resource names. It helps avoid errors working with Stacks, allows the Stack to be deployed repeatedly to the same account/region, etc. It leaves me with problems like this though:

Version: 3

Blueprints:
  DeploymentBucket:
    Template: template1.yaml

Stages:
  dev:
    DeploymentBucketDev:
      Extends: DeploymentBucket
      StackName: myproject-deployment-bucket-dev
      Parameters:
        Environment: dev
        BucketName: myproject-deployment-dev
  prod:
    DeploymentBucketProd:
      Extends: DeploymentBucket
      StackName: myproject-deployment-bucket-prod
      Parameters:
        Environment: prod
        BucketName: myproject-deployment-prod

Because my StackName and Parameters reference the Stage Name, I'm not getting much value from Blueprints. What I would prefer to be able to do is something like this:

Version: 3

Blueprints:
  DeploymentBucket:
    Template: template1.yaml
    StackName: myproject-deployment-bucket-${Stage.StageName}
    Parameters:
      Environment: ${Stage.StageName}
      BucketName: myproject-deployment-${Stage.StageName}

Stages:
  dev:
    DeploymentBucketDev:
      Extends: DeploymentBucket
  prod:
    DeploymentBucketProd:
      Extends: DeploymentBucket

Thoughts?

[Kotaimen]

Hi @alytle, yes that would be a very nice feature to have! (see #38).

@GlieseRay I think current implementation would be a bit difficult to extend into reference values other than outputs, any thoughts? Eg: Setting reference in the blueprint itself would mean your string template implementation will not work properly. Probably need parse the YAML and defer reference for each value.

[alytle]

Sceptre uses Jinja templating within the config files to achieve this, but I'm somewhat opposed to having a full templating engine on top of the config as I find it tends to obfuscate the whole thing and somewhat defeat the point of having a declarative model.

My guess is that we would want some sort of pre-processing step to handle this interpolation.

If there's some agreement on an approach I'm happy to contribute.

[Kotaimen]

@alytle again, contribution are welcome! Using a template engine probably is an overkill ... here's my thought on how this could be done:

• Parse the config and expand blueprints
• For each stack prepare a "ref" structure contains stage/stack name, parameters, outputs, the data structure is lazy loaded
• Find all "value" in each stack which contains the reference syntax (probably using regex)
• Find the reference for each value in the ref data above

Note the implementation must keep compatibility with existing version, so for your "stage name" problem, using ${Stage.StageName} may not work. Either we introduce special syntax (eg: ${::CurrentStage::Name}, or increase the config version and introduce this as a new major release.

what do you think?

[GlieseRay]

@alytle The ${...} is used for stack parameter reference now. So for in config replacement we probably need a new format. But I don't feel that a "ref" structure is needed as long as these identifier is unique in the config. And the replacement could be done before parsing and will not impact blueprint expanding .

Besides, I would prefer keep config parsing simple. it is not practical to accommodate different kinds of string templating in config. We could use a simple script to generate a config file we need.

[Vitiate]

Has there been any follow up on this? This is the biggest shortcoming with cfn-cli at this point in my mind. Even a user-vars like Hashi uses in packer would be nice.