template validation/submit fail if dynamic templateRef is used in nested template

[tczhao]

Pre-requisites

• [X] I have double-checked my configuration
• [X] I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
• [X] I have searched existing issues and could not find a match for this bug
• [ ] I'd like to contribute the fix myself (see contributing guide)

What happened? What did you expect to happen?

When using dynamic template in nested workflow template calls, the validation failed and couldn't submit workflow. calling wftmpl1 -> wftmpl2:dynamic template -> wftmpl3 has issue calling wftmpl2:dynamic template -> wftmpl3 is fine The issue applies to both clusterworkflowtemplate and workflowtemplate

To reproduce the error, run

argo submit --from clusterworkflowtemplate/cwt1  --dry-run -o yaml

This would produce error

FATA[2024-10-17T12:09:45.670Z] Failed to submit workflow: rpc error: code = InvalidArgument desc = templates.main.tasks.cwt1-s1 templates.main2.steps[0].cwt2-s1 template reference cwt3.placeholder-11 not found

Version(s)

latest, e18570c

Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
  name: cwt1
spec:
  entrypoint: main
  templates:
  - name: main
    dag:
      tasks:
      - name: cwt1-s1
        templateRef:
          name: cwt2
          template: main2
          clusterScope: true
---
apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
  name: cwt2
spec:
  entrypoint: main2
  templates:
  - name: main2
    inputs:
      parameters:
      - name: api-template
        value: "abc-abc"
    outputs:
      parameters:
      - name: output-foo
        valueFrom:
          parameter: "{{steps.cwt2-s1.outputs.parameters.foo}}"
    steps:
    - - name: cwt2-s1
        templateRef:
          name: cwt3
          template: "{{inputs.parameters.api-template}}"
          clusterScope: true
---
apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
  name: cwt3
spec:
  templates:
  - name: abc-abc
    container:
      image: busybox
      command: [sleep]
      args: ["1"]
    outputs:
      parameters:
        - name: foo
          valueFrom:
            path: /tmp/abc

Logs from the workflow controller

kubectl logs -n argo deploy/workflow-controller | grep ${workflow}

Logs from in your workflow's wait container

kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

[tczhao]

reverted fix(controller): validation failed when dynamic templateRef is used in nested template #14053 due regression in https://github.com/argoproj/argo-workflows/issues/14165

[rnt-chuet]

Hello there,

Ended up on this thread because of the same error. However, in my use case the Workflow is generated from a CronWorkflow :

1. CronWorkflow references a template A from wftpl2 and passes a template name as arguments.parameters to this template A
2. Template A uses the dynamic template name from the inputs.parameters to call said template from wftpl3
3. wftpl3 defines all valid templates to be called dynamically

If you have ideas whether this will get more support to be fixed, just let me know 🙏

(My Argo controller is in v3.6)

[deynekas]

Hello There,

Experience similar issue after upgrading from 3.5.11 to 3.7.1. When using dynamic templateRef.Name in nested workflow, submit fails with error: template reference placeholder-270.main not found

apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
  name: run-argowf-workflow-v1
spec:
  templates:
  - name: main
    inputs:
      parameters:
      - name: workflowParameters
      - name: workflowName
    dag:
      tasks:
      - name: run-argowf-workflow
        templateRef:
          name: "{{inputs.parameters.workflowName}}"
          template: main
          clusterScope: true
        arguments:
          parameters:
          - name: parameters
            value: "{{inputs.parameters.workflowParameters}}"

same works normally if above workflow submitted directly and not from parent workflow.

[deynekas]

@tczhao is there any chance someone could continue working on fixing this bug? It’s currently a blocker for anyone who use functionality introduced here from upgrading to newer versions of Argo Workflows and Argo Events. Thanks in advance

[tczhao]

Let me take a look this weekend

[kohbis]

What’s the status of this?

[deynekas]

@Joibel when we can expect this change that supposed to fix this bug to be released?