0.3.11 breaking `j2` usage of ENV vars in `docker-compose up`

[ElijahLynn]

Our docker-compose up and ecs-cli up (AWS Elastic Container Service (ECS)) deploys just started failing all of the sudden with:

appserver_1 | Traceback (most recent call last):
appserver_1 | File "/usr/local/bin/j2", line 11, in
appserver_1 | sys.exit(main())
appserver_1 | File "/usr/local/lib/python2.7/dist-packages/j2cli/cli.py", line 206, in main
appserver_1 | sys.argv[1:]
appserver_1 | File "/usr/local/lib/python2.7/dist-packages/j2cli/cli.py", line 186, in render_command
appserver_1 | result = renderer.render(args.template, context)
appserver_1 | File "/usr/local/lib/python2.7/dist-packages/j2cli/cli.py", line 87, in render
appserver_1 | .render(context) \
appserver_1 | File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 1008, in render
appserver_1 | return self.environment.handle_exception(exc_info, True)
appserver_1 | File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 780, in handle_exception appserver_1 | reraise(exc_type, exc_value, tb)
appserver_1 | File "/templates/ssmtp.tpl", line 8, in top-level template code
appserver_1 | rewriteDomain={{ SMTP_FROM_HOSTNAME }}
appserver_1 | jinja2.exceptions.UndefinedError: 'SMTP_RELAY_HOST' is undefined

These are containers that have an entrypoint script that uses j2 to copy a template file into a regular file.

j2 /templates/ssmtp.tpl > /etc/ssmtp/ssmtp.conf

I pinned the version back to 0.3.10 with pip install -Iv j2cli==0.3.10 in the Dockerfile and things are working again.

I recommend reverting https://github.com/kolypto/j2cli/pull/42 (today's release) and pushing this out immediately until a correct patch/PR can be created as it is most likely breaking other builds and deployments. It took me a few hours to arrive here and others may be struggling with why this is happening to them as well.

[kolypto]

@ElijahLynn, your invocation of j2 renders a template from the environment variables that are set elsewhere, so SMTP_RELAY_HOST is defined in that context. Is that right?

[raymondbutcher]

Hi. We're using j2cli in systemd service files to render files using environment variables and it seems 0.3.11 has broken it.

This was working before today but now it's not:

[Service]
EnvironmentFile=/etc/environment
Type=oneshot
ExecStart=/bin/sh -c "j2 <%= @template_path %> > <%= @rendered_path %>"
RemainAfterExit=yes

(It's a Puppet template hence the <%= @rendered_path %> and <%= @rendered_path %>)

We'll just pin the old version :)

[ElijahLynn]

@ElijahLynn, your invocation of j2 renders a template from the environment variables that are set elsewhere, so SMTP_RELAY_HOST is defined in that context. Is that right?

Correct, that var was set elsewhere, from the docker-compose file.

[cmeisinger]

Confirming this as well. I have an entire fleet of containers that won't build now. Same use case as @ElijahLynn.

[kolypto]

Wow, that's unexpected. Sorry. I've pulled the breaking release from pip; will investigate and find out the root cause. Thank you for reporting!

[ElijahLynn]

Wow, that's unexpected. Sorry. I've pulled the breaking release from pip; will investigate and find out the root cause. Thank you for reporting!

Thank you for pulling the breaking release from pip!

[kolypto]

I've made a new release which both has the new functionality, and is backwards-compatible. Please tell me if j2cli==0.3.12b0 works for you? @ElijahLynn @cmeisinger @raymondbutcher

[ElijahLynn]

I've made a new release which both has the new functionality, and is backwards-compatible. Please tell me if j2cli==0.3.12b0 works for you? @ElijahLynn @cmeisinger @raymondbutcher

Just had a look here https://github.com/kolypto/j2cli/commit/26a67e9419d96b7f92871e8b93dba00306c5df0b and that is quite the addition now! Thanks, I'll see if I can test it soon, as I do need to revert the pin anyways. It may be Monday for my schedule. Thanks again

[scaytrase]

Any news here? Is 0.3.12b0 considered stable after a year of beta?