Support dynamic templateRef naming

[pmareke]

Summary

It would be great if we could use variables inside the templateRef field.

This feature will allow us to reuse a template with multiple implementations using inputs parameters.

Use Cases

Below there is an example of what we're after:

templateRef:
  name: '{{inputs.parameters.workflow-template}}'
  template: '{{inputs.parameters.template-name}}'

Thanks!

---

Message from the maintainers:

Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.

[plopezgarcia]

Any progress on this? We are also very much looking forward to this improvement! Thanks in advance

[ehellmann-nydig]

I would like to point out that it's likely that this will have limited usefulness without templating the arguments to pass as well, somehow. Or at least, allowing to template arguments will greatly increase the usefulness of this IMO.

[jonwest]

@ehellmann-nydig Where this would be really valuable is for branching out from a task that has shared parameters. This could be really powerful for running CI against a monorepo, because you could do something like:

     dag:
        tasks:
          # First, pull in the source from the commit:
          - name: pull-source
            templateRef:
              name: pull-source
              template: main
            arguments:
              parameters:
                - name: revision
                  value: "{{ inputs.parameters.revision }}"
          
          # Break workflow down into each of the areas of the monorepo
          # that require CI operations:
          - name: run-component-ci
            templateRef:
              # Source at: workflows/components/<component>/<component>-ci.yaml
              name: "{{ item.component-workflow }}"
              template: main
            dependencies:
              - pull-source
            arguments:
              parameters:
                - name: commit-short-sha
                  valueFrom: "{{ tasks.pull-source.outputs.parameters.commit-short-sha }}"
                - name: comparison-revision
                  value: "{{ inputs.parameters.comparison-revision }}"
              artifacts:
                - name: commit-source
                  from: "{{ tasks.pull-source.outputs.artifacts.commit-source }}"
            withItems:
              - { component-workflow: "component-1-ci" }
              - { component-workflow: "component-2-ci" }
              - { component-workflow: "component-3-ci" }
              - { component-workflow: "component-4-ci" }
              - { component-workflow: "component-5-ci" }

This loop, and this syntax, avoids having to repeat what is essentially the exact same block of configuration five times.

That withItems block is intentionally light, just for the sake of example, but it could include other parameters that would be used within the parameter arguments to allow for triggering multiple similar sub-workflows that share a similar interface without having overly repetitive configuration.

As it is right now, those 27 lines of configuration become 135 lines of configuration, of which 80% of it is repeated unnecessarily.

The only way to do this currently would be to loop through and instead of a task, use the loop to create an entirely new workflow resource, but this falls short because passing artifacts doesn't work, or at the very least is significantly less clear to follow as you need to jump between workflows. It also contributes to workflow bloat—now instead of running and managing the lifecycle for one workflow, you're running and managing six.

[earglass]

I'll give you very simple example on how this would be useful, hopefully this gets some movement..

templateRef:
    name: "production-pipeline-{{worfklow.parameters.version}}"
    template: main
    arguments:
       parameters:
           - name: some_param
              value: 'some value'

All templates following above format have same inputs, just different content and we'd like to dynamically choose which template is executed in step based on provided version. Somehow such simple thing is not currently possible.