Pass `parallelism` via CLI? Error on string vs int

[sebinsua]

Is it possible to pass in parallelism via the command line?

I tried to setup a parameter to do this, but got an error about the value not being a number.

Failed to parse workflow:
error unmarshaling JSON: while decoding JSON: json:
cannot unmarshal string into Go struct field WorkflowSpec.parallelism of type int64

If it's not possible by default to set this for the workflow from the command line, is there a way of coercing a string containing a number into a number?

Edit: Do you have anything like an int filter? I heard you were using Jinja but I can't find it within the codebase.

[simster7]

Is it possible to pass in parallelism via the command line?

Currently not possible.

Do you have anything like an int filter? I heard you were using Jinja but I can't find it within the codebase.

Not as far as I know.

If it's not possible by default to set this for the workflow from the command line, is there a way of coercing a string containing a number into a number?

Could you elaborate a bit on this? Are you trying to run dynamically generated Workflows?

[sebinsua]

If it's not possible by default to set this for the workflow from the command line, is there a way of coercing a string containing a number into a number?

Could you elaborate a bit on this? Are you trying to run dynamically generated Workflows?

I tried this at one point, but it doesn't work:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: global-parallelism-parameter-
spec:
  entrypoint: entrypoint-name
  arguments:
    parameters:
    - name: parallelism
      value: 10
  parallelism: "{{workflow.parameters.parallelism}}"
# ...

I'm not trying to run dynamicly generated workflows. I just wanted to be able to configure the level of parallelism a workflow should be run with at the point of submission.

(FYI, what I'm doing is analogous to this toy workflow but with Cypress E2E test runs instead of computations.)

[simster7]

I see. I don't think we currently support a way to pass in parameters to the Workflow object... only to Template objects within a workflow.

A workaround suggestion I could give you would be to use sed to replace the placeholder value before submitting. Something like this:

$ cat workflow.yaml | sed 's/"{{workflow.parameters.parallelism}}"/10/g' | argo submit --watch -

[dan-katz]

I'm also running into this issue when trying to have parallelism be a parameter of a workflow template

[OmerKahani]

Hi, we also have this problem using Workflow templates, @alexec what more information is needed?

[alexec]

I don't think any information is needed? The label was removed not added.

I think the solution executed in #3610 could work here too.

Would anyone be interested in submitting a PR?

[sasagarw]

I can try working on this and create a PR @alexec

[ksonbol]

Hi, any plan to support this soon? I was expecting at least one of these to work:

• parallelism: "{{ workflow.parameters.parallelism }}"
• parallelism: "{{=asInt(workflow.parameters.parallelism)}}"

I was surprised asInt() didn't fix this casting error, shouldn't it work in all spec levels?

[agilgur5]

I was surprised asInt() didn't fix this casting error, shouldn't it work in all spec levels?

I believe this is because it's quoted (and if it weren't quoted, it would be invalid YAML). As mentioned above, #3610 made some fields able to parse those into ints, but not all fields.

[toandocoffee]

Hi @sasagarw, Do you have any updates on this? Thanks

[wernerwws]

Hi @alexec,

This would be a great feature and would leverage the strengths of Argo Workflows to loop / iterate over Lists and be able to use simpler templates and control the parallelism of those via workflow variables or generated variables / values out of steps.

Is there much missing or is it just an issue with linting the string as an integer?

[agilgur5]

Is there much missing or is it just an issue with linting the string as an integer?

It was mentioned above that the same approach as #3610 could be used here. Right now the spec expects an int; for this feature it would need to also allow a string (which it can convert to an int)

[wesleyscholl]

It would be nice to be able to set parallelism to a variable {{input.parameters.parallelism}}. Especially when trying to set parallelism to a workflow input parameter.

Tested: parallelism: {{=asInt(workflow.parameters.parallelism)}} it doesn't work.

[mustafasezer]

Hi, is there any update on this? I have seen that there is even a PR https://github.com/argoproj/argo-workflows/pull/13368, is this still in progress?