Allow exposing arbitrary structured data to jinja context

[c0m1c5an5]

Feature Request

Allow user to provide additional data to be exposed in the jinja context for changelog generation.

Description

Allow user to pass arbitrary json-formatted data for use in the template interpolation.

Use cases

The most common use case is passing dynamically generated data to the changelog like docker container hashes, built binary checksums, etc.

Possible implementation

Add a cli flag (for example --data) that takes in a json string. Parse that string and add it to the jinja context under the user_data key. This gives the most flexibility to the user. Though, having an ability to pass a json file would also be nice, since there can be cases, where shell command length limit is reached. If this implementation is approved, I can create a PR with these changes.

Alternative solutions

I currently use this workaround:

1. Create a jinja file with a variable definition inside, that consumes a json string.

index_metadata_json="$(jq -cs '.' "${INDEX_METADATA}")"
cat - >.gitlab/semantic-release/.components/image_index_table.j2 <<-EOF
	{%  set image_index_metadata = ${index_metadata_json:-[]} %}
EOF

2. Which is then included in the jinja template:

{%    from 'macros.md.j2' import docker_slugify
%}{%  from 'image_index_table.j2' import image_index_metadata
%}{#
#}{%  if image_index_metadata | length > 0
%}{{    "## Images\n"
}}{{    "| Name | Image |\n"
}}{{    "| ---- | ----- |\n"
}}{%    for meta in image_index_metadata %}{{
          "| %s | `%s:%s` <br> `%s` <br> `%s@%s` |\n" | format(meta.name, meta.repo, docker_slugify(release.version.as_semver_tag()), meta.ref, meta.repo, meta.digest)
}}{%    endfor %}{{
        "\n"
}}{%  endif %}

[codejedi365]

@c0m1c5an5, interesting idea. What if I just expose the json parser function into the context? There is already a read_file filter added to the context and this would convert it to usable JSON. You would need to hardcode the filename into your custom templates though.

[c0m1c5an5]

@codejedi365 Yes, that provides even more options for the end user than I intended. This is probably the best option.

[pdcastro]

Add a cli flag (for example --data) that takes in a json string.

Indeed, I have come across this issue while looking for a way to expose data from CI environment variables to the context in Jinja templates. Reading JSON from a file works for sure, 👍 but in simple cases like trying to load a short string of comma-separated items such as 'item1,item2,..., it would be rather convenient to have a solution like:

# CI shell script
export MY_ITEMS='item1,item2,item3'
./venv/bin/semantic-release version --env MY_ITEMS --changelog --vcs-release ...

or

./venv/bin/semantic-release version --env MY_ITEMS='item1,item2,item3' --changelog --vcs-release ...

And then in the Jinja template, something along the lines of:

{% for item in context.env.get('MY_ITEMS', '').split(',') %}
    * This item’s value is {{ item }}.
{% endfor %}

Above I've used Python’s syntax for the dict.get and str.split methods, which should be available in Jinja according to Python Methods in Jinja. Related StackOverflow post: Accessing OS environment variables from Jinja2 template.

Neither JSON nor files are needed in this example. This would be an additional solution to the alternative of reading JSON from files.

Either way, great stuff! 👍🚀

[pdcastro]

semantic-release version --env ...

Similarly to the docker run --env option: https://docs.docker.com/reference/cli/docker/container/run/#env

[c0m1c5an5]

At this point I suggest porting core ansible filters and lookups to python-semantic-release and exposing them to the environment. It contains file read, environment lookup and a few other utilities. PS. I don't mean all of them, but only the most useful.

[codejedi365]

@c0m1c5an5, thanks for the example. I started reviewing them and will see how many I add from ansible's implementation.

[codejedi365]

@pdcastro, thanks for adding to the discussion of what you would like too. I probably will not use a CLI option like docker but likely something similar to the build_command_env option (changelog.context_env) and the env get added to the context as your example asks. The reason for having it specified is similar to docker in defining the env values to pass. It prevents an arbitrary passing of secrets into a context which are hard for a program to determine which the user would never intentionally pass.

[github-actions[bot]]

It has been 90 days since the last update on this confirmed issue. @python-semantic-release/team can you provide an update on the status of this issue?