Templating with Jinja2/YAML

[tristanpemble]

I'm gonna start off by saying that I'm not sure if there's a problem or a solution here. I guess I just wanted to have a discussion about the semantics of templating with Forge. Right now, templates are considered Jinja2 first, and then YAML. When I asked about this on Gitter, @rhs mentioned he wasn't sure if this was the right choice, it just was how he had implemented it at first. I think it might work out to be the best implementation.

• What are the advantages of each approach? (YAML file with Jinja2 values, or a Jinja2 template rendering a YAML file)
• What existing examples of templated YAML are there?
  • Ansible (YAML file with Jinja2 values)
  • ???

I originally typed out a long example situation I ran into but it ended up not being relevant, so I boiled this issue down into some discussion points that might be good to have.

[tristanpemble]

Below are just notes of a situation I ran into..

Say I have a flag in my service.yaml, like useSsl. I want to toggle it with an environment variable. My first guess is to do something like:

# service.yaml
useSsl: {{ env.SSL or false }}

# k8s/ingress.yaml
# ...
metadata:
  annotations:
    kubernetes.io/tls-acme: {{ service.useSsl }}
    kubernetes.io/ingress.allow-http: {{ not service.useSsl }}
    ingress.kubernetes.io/force-ssl-redirect: {{ service.useSsl }}
# ...

# rendered at work/k8s/ingress.yaml
# ...
metadata:
  annotations:
    kubernetes.io/tls-acme: True
    kubernetes.io/ingress.allow-http: False
    ingress.kubernetes.io/force-ssl-redirect: True
# ...

The problem here is that Kubernetes actually wants strings "true" and "false" so this manifest doesn't work.

So then I try {{ 'true' if service.useSsl 'false' }}, but that renders as true, still, even though it's a string in Jinja2. Which totally makes sense, once you run into it. This is a Jinja2 template, rendering text, that is later interpreted as YAML. You have to keep that in mind, but the mismatches can be a little weird.

So what I end up doing is "{{'true' if service.useSsl 'false'}}", which finally renders "true"; it looks very strange.

It can give you a lot of flexibility, however, with conditional blocks, since you can sort of just form a YAML document however you want:

# k8s/ingress.yaml
# ...
metadata:
    {% if service.useSsl %}
    kubernetes.io/tls-acme: 'true'
    kubernetes.io/ingress.allow-http: 'false'
    ingress.kubernetes.io/force-ssl-redirect: 'true'
    {% endif %}

[plombardi89]

Ansible 1.x had the same issue and I've been bitten and frustrated by it countless times in the past. Not sure about 2.x but I assume given it still uses Jinja2 and YAML that it has this issue as well. YAML is partially at fault here for allowing unquoted strings which are ambiguous in many cases. It's very frustrating.

You can do this:

# k8s/ingress.yaml
# ...
metadata:
  annotations:
    kubernetes.io/tls-acme: "{{ service.useSsl | lower }}"
    kubernetes.io/ingress.allow-http: "{{ not service.useSsl | lower }}"
    ingress.kubernetes.io/force-ssl-redirect: "{{ service.useSsl | lower }}"
# ...

[plombardi89]

Now that I think about it I think Ansible 2 gives YAML precedence over Jinja2. It does some fancy node walking and interpolates as it moves through values contained in the YAML tree. I remember another issue specific to Ansible handling of YAML and Jinja2 where "key: {{ somevar }}" is considered an error because it gets parsed as a YAML/JSON object embedded in the document first rather than as a Jinja2 templating construct.

I think no matter which order you go there is going to be pain.

[rhs]

Thanks for filing this issue. Great discussion so far. At a minimum I will make sure any gotchas make it into the docs. A few thoughts to add.

1. In this particular case I believe there is a gotcha here that doesn't involve jinja2 or yaml. I think the kubernetes schema is actually defined somewhat inconsistently with respect to annotations. Unlike elsewhere in your manifest the values are required to be strings. While I can see why this might make sense from an API perspective (annotations are a freeform map), this turns into a gotcha when you have annotations in the same manifest with properly typed values. For example a port number in an annotation needs to be quoted, but a port number elsewhere does not. I think you could get stung by this even if you supply your manifests as plain old JSON. I did read somewhere that there were mumblings about allowing annotation values to hold arbitrary JSON, so this gotcha may go away in the future.

2. Validation and error reporting are worth thinking about as well. While working on some of the other open issues, I've added a schema yesterday for service.yaml as it is growing large enough to warrant it at this point. My goal in general is to be able to reference the filename and line number along with the error message whenever there is a problem. On the surface this all seems harder to piece together if the input is actually a yaml tree of jinja snippets rather than a jinja template that produces yaml, i.e. you just have a fundamentally more complex processing model in the latter case:

service.yaml.j2 --(render)--> service.yaml --(yaml parse, schema validate)--> ...

vs

service.j2.yaml --(yaml parse without validation)--> *tree-of-j2-snippets* --(traverse tree in some order and render snippets)--> service.yaml -->(yaml parse, schema validate) --> ...

[tristanpemble]

I think a lot of where friction might come from is typecasting and being forced to use strings, but I just realized something. since valid JSON is valid YAML, a lot of that can be dealt with by just using a tojson filter:

somekey: {{ service.complicatedConfigObject | tojson }}

stringBoolean: {{ service.useSsl | lower | tojson }}

in the end I think the current method is the safest approach, but any of the friction points can be smoothed over with, like you suggested, helpful validation, and good documentation