Add a substitution pass

[rillian]

I'd like there to be a way to inject secrets or other per-instance values into an ignition file when fcct is run.

Use case

An application config file contains numerous settings, and so should be kept in version control. However, one of those settings is an authorization token for accessing privileged data. That token needs to be kept secret, so a repository containing it cannot be published. It should also be different for each instance, leading to either churn or duplication.

Instead, it should be possible to put a some kind of placeholder in the config file so the repository can be public, and substitute a particular value for the authorization token from some external source when the instance is created.

Implementation suggestions

I'd like the transpiler to apply some kind of template or variable expansion pass. So I could have this server.yaml in a public repository:

storage:
  files:
    - path: /etc/app/config
      contents:
        inline: |
          memory 50M
          queue 10
          fetch https://example.com/jobs
          token {{ auth_token }}
          timeout 200

And then fcct would substitute the actual value for auth_token when it creates server.ign. The secret could be taken from the environment or from another (yaml?) dictionary file passed as an additional argument:

$ fcct --strict --pretty --var secrets.yaml server.yaml > server.ign

This would also be useful for any content that varies by instance, like upstream urls or resource limits. It might make more sense to call it local.yaml in that case.

A possible enviroment-based substitution:

$ auth_token=$(fetch_app_token.py --expires +90d)
$ fcct --strict --pretty server.yaml > server.ign
$ unset auth_token

Of course one can achieve this by running either the source yaml or the target ignition files through an external template engine (which might just be ad-hoc sed commands) as part of a wrapper or deployment script. However it's a common enough use case that I think it's worth supporting a standard solution for it.

Issues

• IIRC go has some handlebarsesque template library, which is why I suggested the {{ var }} syntax above. It looks like Azure pipelines use shell-style $(var) expansion instead. Yaml already has in-file variable substitution (aliases) but I rarely see anyone use it. That might enable implementation just by concatenating the file before parsing.

  Either way, care should probably be taken with how the macro expansion interacts with surrounding context. Must the substitution end up as a single value? Auto-escaped? Or can it include arbitrary sub-trees? Arbitrary file sections? I don't know what the correct choice is.

• A number of other solutions use encrypted values instead, so only a single key needs to be injected, which then decrypts all the secret values. This is simpler in some ways, and allows a kind of change-tracking for values themselves. See Travis CI and Ansible Vaults.

[lucab]

An application config file contains numerous settings, and so should be kept in version control. However, one of those settings is an authorization token for accessing privileged data. That token needs to be kept secret, so a repository containing it cannot be published. It should also be different for each instance, leading to either churn or duplication.

@rillian thanks for the report. Just wondering, did you see the configuration entry to source local filesystem resources (https://github.com/coreos/fcct/pull/99)? It seems to solve the usecase described above (committing the config, but not the secret file) without introducing a full templating language.

[rillian]

Thanks for taking a look, @lucab. The local source for the storage.files.contents is great! However, it only helps when the secret can be isolated in its own file on the target system. The use case is having a single value inside a larger file which should otherwise be tracked in commits.

[waisbrot]

In past projects, I have added a step myself to run the code through j2cli.

Advantages:

• Jinja2 is a powerful template language (You probably won't run into situations where you need a hack on top of this)
• Existing template language so examples are easy to find
• Doesn't add complexity to the tool that you want to template (more Unix-y philosophy)

Disadvantages:

• You need to have some kind of automation (makefile, shell-script, etc). Usually if it's complex enough for me to need a template, it's complex enough for me to have some automation.
• Jinja2 needs Python. Inconvenient if the machine you're deploying from is bare-bones
• Not under control of fcct. With extra work, you could do something like Terraform's secret variables, where they don't show up in error output?

[jasonmccallister]

Just adding my two cents, having a YAML file with variables and running butane with a file or list of variables (similar to Terraform templatefile) would be really helpful. Right now we are using Terraform templatefile -> YAML -> Butane (CT provider) to generate Ignition configurations... This feature would allow testing locally without involving Terraform.

Something like butane --strict --pretty --var-file vars.json config.yaml?

[jhertum]

envsubst is available in most distros, so envsubst < server.yaml | butane > server.ign should replace all the exported environment variables.

[rillian]

Thanks, I didn't know about envsubst, that's a useful work around. Looks like it's part of gettext so likely to be available on developer images.

[drjrkuhn]

yaml doesn't play very well with multiline envsubst variables. For example, I need to add a couple of network interface configurations using /etc/NetworkManager/system-connections/ensXXX.nmconnection files in butane. I can't pull a multiline network config from a bash variable unless the indentation perfectly matches the yaml file.