`inputs` or `workflow` is nil when used in sprig expression that has more than 1 parameter (`withParam` loop)

[aaaaahaaaaa]

Checklist

• [x] Double-checked my configuration.
• [x] Tested using the latest version.
• [x] Used the Emissary executor.

Summary

What happened/what you expected to happen?

With the following context:

• within a withParam loop
• in argument.parameter.value
• using a sprig function that as more than 1 parameter (e.g. sprig.replace)
• referencing workflow.parameters.paramX or inputs.parameters.paramX

Then, the following error is thrown:

  Warning  WorkflowFailed   3s    workflow-controller  failed to evaluate expression: cannot fetch parameters from <nil> (1:23)
 |  sprig.replace(inputs.parameters.paramX, 'YYY', 'Replace XXX') 
 | ......................^

If the expression uses a sprig function with 1 parameter, then the error doesn't occur.

What version are you running?

3.3.0

Diagnostics

Paste the smallest workflow that reproduces the bug. We must be able to run the workflow.

---
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: my-template
  namespace: argo
spec:
  entrypoint: execute

  arguments:
    parameters:
      - name: paramX
        value: XXX
      - name: paramY
        valueFrom:
          configMapKeyRef:
            name: my-config
            key: my-key

  templates:
    - name: execute
      steps:
        - - name: other-template
            templateRef:
              name: other-template
              template: execute
            withParam: "{{workflow.parameters.paramY}}"
            arguments:
              parameters:
                - name: my-param
                  value: "{{= sprig.replace(workflow.parameters.paramX, "YYY", "Replace XXX" }}"

All the following are however working and are producing the expected values:

              parameters:
                - name: my-param
                  value: "{{ workflow.parameters.paramX }}"

              parameters:
                - name: my-param
                  value: "{{= workflow.parameters.paramX }}"

              parameters:
                - name: my-param
                  value: "{{= sprig.trim(workflow.parameters.paramX) }}"

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

[aaaaahaaaaa]

Could be related to https://github.com/argoproj/argo-workflows/issues/6790? @alexec

[sarabala1979]

@aaaaahaaaaa Is it only happening when you are using input.parameters.xxxx?

[aaaaahaaaaa]

@sarabala1979 Or workflow.parameters as mentioned above.

[aaaaahaaaaa]

@sarabala1979 I did some more testing and I think I drilled down to a more specific diagnostic that my example above doesn't show.

The problem seems to occur when both worflow.parameters.XXX and item.YYY are used next to each other as parameters of the same function (as part of a withParam loop).

Examples:

WORKS

{{= sprig.replace('what ever', workflow.parameters.XXX, workflow.parameters.XXX) }}

WORKS

{{= sprig.replace('what ever', item.YYY, item.YYY) }}

ERROR cannot fetch parameters from <nil>...

{{= sprig.replace('what ever', workflow.parameters.XXX, item.YYY) }}

This seems really strange but I believe there's really a bug here.

[tacf]

Hi, bumped into an issue that i think it may relate to this.

There's a weird behaviour when using both workflow. and inputs. in an expression. In my case i'm trying to dynamically set a mutex, below i'll leave the examples and the calculated mutex value to see if it helps trying to pin point this.

workflow.parameters.XXX = true
inputs.parameters.YYY = false

"{{=workflow.parameters.XXX}}" # argo/Mutex/true
"{{=workflow.parameters.XXX == 'true' ? workflow.uid : workflow.parameters.XXX }}" # argo/Mutex/d003efcc-7817-4dc0-b435-5c98640ac358
"{{=inputs.parameters.YYY == 'false' ? workflow.uid : 'falseee' }}" # argo/Mutex/{{=inputs.parameters.YYY == 'false' ? workflow.uid : 'falseee' }}
"{{inputs.parameters.YYY}}" # argo/Mutex/false
"{{=inputs.parameters.YYY}}" # argo/Mutex/false
"{{=workflow.parameters.XXX == 'true' and inputs.parameters.YYY == 'false' ? 'true' : 'false'}}" # argo/Mutex/{{=workflow.parameters.XXX == 'true' and inputs.parameters.YYY == 'false' ? 'true' : 'false'}}
"{{=inputs.parameters.YYY == 'false' and workflow.parameters.XXX == 'true' ? 'true' : 'false'}}" # argo/Mutex/{{=inputs.parameters.YYY == 'false' and workflow.parameters.XXX == 'true' ? 'true' : 'false'}}
"{{=inputs.parameters.YYY == 'false' and inputs.parameters.YYY == 'true' ? 'true' : 'false'}}" # argo/Mutex/false
"{{=workflow.parameters.XXX == 'true' and workflow.parameters.XXX == 'false' ? 'true' : 'false'}}" # argo/Mutex/false 

As you can see in the case that both workflow. and inputs. is used the mutex value goes as the literal string of the expression, and according to the tests i did is not just a visual thing (it could be either an UI / Logs issue no rendering the calculated value) the workflow is actually locking on the expression string.

Let me know if you find this is related or should i move this to a new issue.

[alexec]

I don't think you can use "and", you should use "&&".

[tacf]

Unless this is a custom configuration on argo side Expr does support this

Edit: This can easily be tested by something like {{= 'X' == 'X' and 'Y' == 'Y' ? ... }} which works just fine.

[alexec]

The example workflow is invalid.

[alexec]

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: my-wf-
spec:
  entrypoint: main

  arguments:
    parameters:
      - name: XXX
        value: "true"
      - name: YYY
        value: "false"

  templates:
    - name: main
      inputs:
        parameters:
          - name: XXX
            value: "{{workflow.parameters.XXX}}"
          - name: YYY
            value: "{{workflow.parameters.YYY}}"
      container:
        image: argoproj/argosay:v2
        command:
          - sh
          - -c
        args:
          - |
            echo "{{=workflow.parameters.XXX}}" # argo/Mutex/true
            echo "{{=workflow.parameters.XXX == 'true' ? workflow.uid : workflow.parameters.XXX }}" # argo/Mutex/d003efcc-7817-4dc0-b435-5c98640ac358
            echo "{{=inputs.parameters.YYY == 'false' ? workflow.uid : 'falseee' }}" # argo/Mutex/{{=inputs.parameters.YYY == 'false' ? workflow.uid : 'falseee' }}
            echo "{{inputs.parameters.YYY}}" # argo/Mutex/false
            echo "{{=inputs.parameters.YYY}}" # argo/Mutex/false
            echo "{{=workflow.parameters.XXX == 'true' and inputs.parameters.YYY == 'false' ? 'true' : 'false'}}" # argo/Mutex/{{=workflow.parameters.XXX == 'true' and inputs.parameters.YYY == 'false' ? 'true' : 'false'}}
            echo "{{=inputs.parameters.YYY == 'false' and workflow.parameters.XXX == 'true' ? 'true' : 'false'}}" # argo/Mutex/{{=inputs.parameters.YYY == 'false' and workflow.parameters.XXX == 'true' ? 'true' : 'false'}}
            echo "{{=inputs.parameters.YYY == 'false' and inputs.parameters.YYY == 'true' ? 'true' : 'false'}}" # argo/Mutex/false
            echo "{{=workflow.parameters.XXX == 'true' and workflow.parameters.XXX == 'false' ? 'true' : 'false'}}" # argo/Mutex/false 

✗ argo submit --log tmp.yaml 
Name:                my-wf-h4p5d
Namespace:           argo
ServiceAccount:      unset (will run with the default ServiceAccount)
Status:              Pending
Created:             Mon Apr 04 07:36:02 -0700 (now)
Progress:            
Parameters:          
  XXX:               true
  YYY:               false

This workflow does not have security context set. You can run your workflow pods more securely by setting it.
Learn more at https://argoproj.github.io/argo-workflows/workflow-pod-security-context/
my-wf-h4p5d: time="2022-04-04T14:36:04.207Z" level=info msg="capturing logs" argo=true
my-wf-h4p5d: true
my-wf-h4p5d: c9f7ac42-fa9d-44fe-8694-7479965a3a32
my-wf-h4p5d: c9f7ac42-fa9d-44fe-8694-7479965a3a32
my-wf-h4p5d: false
my-wf-h4p5d: false
my-wf-h4p5d: true
my-wf-h4p5d: true
my-wf-h4p5d: false
my-wf-h4p5d: false
```

[alexec]

You're correct, and works. I learnt something.

I get a different result to you.

[aaaaahaaaaa]

@alexec What about the original issue?

[alexec]

Could you please re-state the problem?

[aaaaahaaaaa]

@alexec You mean the original post is unclear?

[aaaaahaaaaa]

The gist is, considering the following workflow:

---
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: my-template
  namespace: argo
spec:
  entrypoint: execute

  arguments:
    parameters:
      - name: X
        value: xxx
      - name: Y
        valueFrom:
          configMapKeyRef:
            name: my-config
            key: my-key

  templates:
    - name: execute
      steps:
        - - name: other-template
            templateRef:
              name: other-template
              template: execute
            withParam: "{{workflow.parameters.Y}}"
            arguments:
              parameters:
                - name: my-param
                  value: "{{= sprig.replace('what ever', workflow.parameters.X, item.Y) }}"

Then the following error is raised:

  Warning  WorkflowFailed   3s    workflow-controller  failed to evaluate expression: cannot fetch parameters from <nil> (1:23)
 |  sprig.replace('what ever', workflow.parameters.paramX, item.Y) 
 | ............................^

The problem seems to specifically occur when both worflow.parameters.X and item.Y are used next to each other as parameters of the same function.

And again, the following ARE working:

{{= sprig.replace('what ever', workflow.parameters.X, workflow.parameters.X) }}

{{= sprig.replace('what ever', item.Y, item.Y) }}

Which is why I believe there's clearly a bug here.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[aaaaahaaaaa]

@alexec Would you confirm that looks like a bug to you?

[rkt2spc]

We also have this problem @alexec @aaaaahaaaaa It's not a problem with sprig. It's a problem with Argo variables expression in general

      steps:
        - - name: loop
            template: script
            arguments:
              parameters:
                - name: ok
                  value: "{{ workflow.parameters.some_param }} : {{ item }}"
                - name: ok2
                  value: "{{= asInt(workflow.parameters.iterations) }}"
                - name: ok3
                  value: "{{= asInt(item) }}"
                - name: not_ok
                  value: "{{= asInt(workflow.parameters.some_param) + asInt(item) }}"
            withSequence:
              count: "5"

[alexec]

It's not a problem with sprig. It's a problem with Argo variables expression in general

Could you please explain more?

[rkt2spc]

@alexec

This should work, but it doesn't. I'm using Argo Workflow v3.3.3

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: debug-
spec:
  arguments:
    parameters:
      - name: message
        value: "hello"

  entrypoint: entrypoint

  templates:
    - name: entrypoint
      steps:
        - - name: broken
            template: speak
            arguments:
              parameters:
                - name: message
                  value: "{{= workflow.parameters.message + ' ' + item }}"
            withSequence:
              count: "3"

    - name: speak
      inputs:
        parameters:
          - name: message
      script:
        image: python:alpine
        command: ["python"]
        source: |-
          print("{{ message }}")

Expectations

{{= workflow.parameters.message + ' ' + item }} should evaluated to hello 0 hello 1 hello 2 for each fanned-out instance of the step broken

Reality

I got

Change {{= workflow.parameters.message + ' ' + item }} to {{ workflow.parameters.message }} {{ item }} and it works as expected.

But in my use-case I need to do some advanced evaluation using both the workflow param and the loop param e.g {{= asInt(workflow.parameters.message) * asInt(item) }}

[gg314]

This might be a long shot, but @aaaaahaaaaa and @rocketspacer, could you try using workflow.parameters['message-param'] instead of workflow.parameters.message-param?

---

edit: this issue is limited to hyphenated parameters

[aaaaahaaaaa]

This might be a long shot, but @aaaaahaaaaa and @rocketspacer, could you try using workflow.parameters['message'] instead of workflow.parameters.message?

Already tried. Same outcome.

[gg314]

That's a shame. I thought I was having the same problem but switching to this syntax fixed it. There is definitely something strange about the parser...

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is a mentoring request, please provide an update here. Thank you for your contributions.

[zjgemi]

I think the problem is that the workflow contexts (workflow.xxx, inputs.xxx, steps.xxx or tasks.xxx) and the loop context (item) cannot be introduced simultaneously for an expression (starts with {{=).

[stale[bot]]

This issue has been closed due to inactivity. Feel free to re-open if you still encounter this issue.

[aaaaahaaaaa]

Up

[ChillAndImprove]

Up

[tooptoop4]

Up

[philippeboyd]

@alexec any news on this? I can replicate the issue with a ternary operator

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: debug-
spec:
  arguments:
    parameters:
      - name: default
        value: "HELLO"
      - name: message
        value: |
          [
            {"a" :1},
            {"a" :1,"b": 2}
          ]

  entrypoint: entrypoint

  templates:
    - name: entrypoint
      steps:
        - - name: broken
            template: speak
            arguments:
              parameters:
                - name: message
                  value: "{{=item.b != nil ? item.b : workflow.parameters.default}}"
            withParam: "{{workflow.parameters.message}}"

    - name: speak
      inputs:
        parameters:
          - name: message
      script:
        image: python:alpine
        command: ["python"]
        source: |-
          print("{{ inputs.parameters.message }}")

However, replacing the value with value: "{{=item.b != nil ? item.b : 'BYE'}}" works. Why can't we reference any workflow.parameters?

[kaaquist]

up