jsonpath behavior on an array is invalid

[GlobeFishNG]

Pre-requisites

• [X] I have double-checked my configuration
• [ ] I can confirm the issues exists when I tested with :latest
• [ ] I'd like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?

The issue is as below.

• 1st step generated a JSON output

  {
   "test": [{
       "a": "100"
   },
   {
       "a": "101"
   }]
  }
  

• 2nd step used jsonpath expression {{=jsonpath(tasks.hello.outputs.parameters.out, '$.test[*].a')}} to extract all values of the test.a property. And the expectation is ["100","101"], but got [100 101]

Here is the log

kl hello-world-2wxd9-world-2103338487
time="2023-04-14T04:57:37.629Z" level=info msg="capturing logs" argo=true
[100 101]
time="2023-04-14T04:57:38.633Z" level=info msg="sub-process exited" argo=true error="<nil>"

Version

v3.3.4

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: Workflow                  
metadata:
  generateName: hello-world-    
spec:
  serviceAccountName: workflow
  entrypoint: test          
  templates:
    - name: test
      dag:
        tasks:
          - name: hello
            template: hello
          - name: world
            template: world
            depends: hello
            arguments:
              parameters:
                - name: in
                  value: "{{=jsonpath(tasks.hello.outputs.parameters.out, '$.test[*].a')}}"
    - name: hello              
      outputs:
        parameters:
          - name: out
            valueFrom:
              path: /mnt/out/json
      container:
        image: alpine
        command: [ sh, -c ]
        args:
          - 'echo "{\"test\": [{\"a\": \"100\"},{\"a\": \"101\"}]}" | tee /mnt/out/json'
        resources: # limit the resources
          limits:
            memory: 32Mi
            cpu: 100m
        volumeMounts:
            - name: out
              mountPath: /mnt/out
      volumes:
        - name: out
          emptyDir: { }
    - name: world              
      inputs:
        parameters:
          - name: in
      container:
        image: alpine
        command: [ sh, -c ]
        args:
          - 'echo {{inputs.parameters.in}}'
        resources: # limit the resources
          limits:
            memory: 32Mi
            cpu: 100m

Logs from the workflow controller

time="2023-04-14T04:57:23.091Z" level=info msg="Processing workflow" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:23.102Z" level=info msg="Updated phase  -> Running" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:23.103Z" level=info msg="DAG node hello-world-2wxd9 initialized Running" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:23.103Z" level=info msg="All of node hello-world-2wxd9.hello dependencies [] completed" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:23.103Z" level=info msg="Pod node hello-world-2wxd9-2766627855 initialized Pending" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:23.144Z" level=info msg="Created pod: hello-world-2wxd9.hello (hello-world-2wxd9-hello-2766627855)" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:23.145Z" level=info msg="TaskSet Reconciliation" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:23.145Z" level=info msg=reconcileAgentPod namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:23.169Z" level=info msg="Workflow update successful" namespace=app phase=Running resourceVersion=242326289 workflow=hello-world-2wxd9
time="2023-04-14T04:57:33.091Z" level=info msg="Processing workflow" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:33.091Z" level=info msg="Task-result reconciliation" namespace=app numObjs=1 workflow=hello-world-2wxd9
time="2023-04-14T04:57:33.091Z" level=info msg="task-result changed" namespace=app nodeID=hello-world-2wxd9-2766627855 workflow=hello-world-2wxd9
time="2023-04-14T04:57:33.092Z" level=info msg="node changed" namespace=app new.message= new.phase=Succeeded new.progress=0/1 nodeID=hello-world-2wxd9-2766627855 old.message= old.phase=Pending old.progress=0/1 workflow=hello-world-2wxd9
time="2023-04-14T04:57:33.093Z" level=info msg="All of node hello-world-2wxd9.world dependencies [hello] completed" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:33.094Z" level=info msg="Pod node hello-world-2wxd9-2103338487 initialized Pending" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:33.148Z" level=info msg="Created pod: hello-world-2wxd9.world (hello-world-2wxd9-world-2103338487)" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:33.148Z" level=info msg="TaskSet Reconciliation" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:33.148Z" level=info msg=reconcileAgentPod namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:33.170Z" level=info msg="Workflow update successful" namespace=app phase=Running resourceVersion=242326532 workflow=hello-world-2wxd9
time="2023-04-14T04:57:33.176Z" level=info msg="cleaning up pod" action=labelPodCompleted key=app/hello-world-2wxd9-hello-2766627855/labelPodCompleted
time="2023-04-14T04:57:43.150Z" level=info msg="Processing workflow" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.151Z" level=info msg="Task-result reconciliation" namespace=app numObjs=2 workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.151Z" level=info msg="task-result changed" namespace=app nodeID=hello-world-2wxd9-2103338487 workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.151Z" level=info msg="node changed" namespace=app new.message= new.phase=Succeeded new.progress=0/1 nodeID=hello-world-2wxd9-2103338487 old.message= old.phase=Pending old.progress=0/1 workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.152Z" level=info msg="Outbound nodes of hello-world-2wxd9 set to [hello-world-2wxd9-2103338487]" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.152Z" level=info msg="node hello-world-2wxd9 phase Running -> Succeeded" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.152Z" level=info msg="node hello-world-2wxd9 finished: 2023-04-14 04:57:43.152962975 +0000 UTC" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.152Z" level=info msg="Checking daemoned children of hello-world-2wxd9" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.153Z" level=info msg="TaskSet Reconciliation" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.153Z" level=info msg=reconcileAgentPod namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.153Z" level=info msg="Updated phase Running -> Succeeded" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.153Z" level=info msg="Marking workflow completed" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.153Z" level=info msg="Marking workflow as pending archiving" namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.153Z" level=info msg="Checking daemoned children of " namespace=app workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.159Z" level=info msg="cleaning up pod" action=deletePod key=app/hello-world-2wxd9-1340600742-agent/deletePod
time="2023-04-14T04:57:43.213Z" level=info msg="Workflow update successful" namespace=app phase=Succeeded resourceVersion=242326769 workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.333Z" level=info msg="archiving workflow" namespace=app uid=de775e4c-baf0-4783-b7d7-9a7e231925a4 workflow=hello-world-2wxd9
time="2023-04-14T04:57:43.339Z" level=info msg="cleaning up pod" action=labelPodCompleted key=app/hello-world-2wxd9-world-2103338487/labelPodCompleted

Logs from in your workflow's wait container

time="2023-04-14T04:57:35.368Z" level=info msg="Starting Workflow Executor" version=v3.4.4
time="2023-04-14T04:57:35.372Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
time="2023-04-14T04:57:35.372Z" level=info msg="Executor initialized" deadline="0001-01-01 00:00:00 +0000 UTC" includeScriptOutput=false namespace=app podName=hello-world-2wxd9-world-2103338487 template="{\"name\":\"world\",\"inputs\":{\"parameters\":[{\"name\":\"in\",\"value\":\"[100 101]\"}]},\"outputs\":{},\"metadata\":{},\"container\":{\"name\":\"\",\"image\":\"alpine\",\"command\":[\"sh\",\"-c\"],\"args\":[\"echo [100 101]\"],\"resources\":{\"limits\":{\"cpu\":\"100m\",\"memory\":\"32Mi\"}}},\"archiveLocation\":{\"archiveLogs\":true,\"s3\":{\"endpoint\":\"minio.common:9000\",\"bucket\":\"argo-workflows-artifacts\",\"insecure\":true,\"accessKeySecret\":{\"name\":\"minio-secret\",\"key\":\"AWS_ACCESS_KEY_ID\"},\"secretKeySecret\":{\"name\":\"minio-secret\",\"key\":\"AWS_SECRET_ACCESS_KEY\"},\"key\":\"my-artifacts/2023/04/14/hello-world-2wxd9/hello-world-2wxd9-world-2103338487\"}}}" version="&Version{Version:v3.4.4,BuildDate:2022-11-29T16:49:53Z,GitCommit:3b2626ff900aff2424c086a51af5929fb0b2d7e5,GitTag:v3.4.4,GitTreeState:clean,GoVersion:go1.18.8,Compiler:gc,Platform:linux/amd64,}"
time="2023-04-14T04:57:35.372Z" level=info msg="Starting deadline monitor"
time="2023-04-14T04:57:39.373Z" level=info msg="Main container completed" error="<nil>"
time="2023-04-14T04:57:39.373Z" level=info msg="No Script output reference in workflow. Capturing script output ignored"
time="2023-04-14T04:57:39.373Z" level=info msg="No output parameters"
time="2023-04-14T04:57:39.373Z" level=info msg="No output artifacts"
time="2023-04-14T04:57:39.373Z" level=info msg="S3 Save path: /tmp/argo/outputs/logs/main.log, key: my-artifacts/2023/04/14/hello-world-2wxd9/hello-world-2wxd9-world-2103338487/main.log"
time="2023-04-14T04:57:39.373Z" level=info msg="Creating minio client using static credentials" endpoint="minio.common:9000"
time="2023-04-14T04:57:39.373Z" level=info msg="Saving file to s3" bucket=argo-workflows-artifacts endpoint="minio.common:9000" key=my-artifacts/2023/04/14/hello-world-2wxd9/hello-world-2wxd9-world-2103338487/main.log path=/tmp/argo/outputs/logs/main.log
time="2023-04-14T04:57:39.382Z" level=info msg="Save artifact" artifactName=main-logs duration=9.06498ms error="<nil>" key=my-artifacts/2023/04/14/hello-world-2wxd9/hello-world-2wxd9-world-2103338487/main.log
time="2023-04-14T04:57:39.382Z" level=info msg="not deleting local artifact" localArtPath=/tmp/argo/outputs/logs/main.log
time="2023-04-14T04:57:39.382Z" level=info msg="Successfully saved file: /tmp/argo/outputs/logs/main.log"
time="2023-04-14T04:57:39.400Z" level=info msg="Create workflowtaskresults 201"
time="2023-04-14T04:57:39.401Z" level=info msg="Deadline monitor stopped"
time="2023-04-14T04:57:39.401Z" level=info msg="stopping progress monitor (context done)" error="context canceled"
time="2023-04-14T04:57:39.401Z" level=info msg="Alloc=6935 TotalAlloc=13476 Sys=34258 NumGC=4 Goroutines=9"

[sarabala1979]

@GlobeFishNG Can you try to reproduce on 3.4.7?

[Joibel]

I get the same result on 3.4.7

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is a mentoring request, please provide an update here. Thank you for your contributions.

[agilgur5]

We may want to retry this since https://github.com/argoproj/pkg/pull/486 / #11704 updated to a newer and more comprehensive JSONPath library. That was released in 3.5.0-rc2