Bash script example?

[Trollgeir]

I've been struggling a bit with the API, and I can't quite seem to figure out how to run a generic script with Hera (independent of python). Here's an example from the argo workflows running a bash script.

[Trollgeir]

def hello():
    'echo hello world'

ws = WorkflowService(host="XXX", namespace="YYY", token=token)
w = Workflow("test", ws, namespace=ws._namespace)
t = Task("t", hello, command=['bash'])
w.add_task(t)
w.create()

Results in:

...
'script': {'command': ['bash'],
                                    'image': 'python:3.7',
                                    'imagePullPolicy': 'Always',
                                    'name': 't',
                                    'resources': {'limits': {'cpu': '1',
                                                             'memory': '4Gi'},
                                                  'requests': {'cpu': '1',
                                                               'memory': '4Gi'}},
                                    'source': "'echo hello world'\n"}}]},
...                                    

The extra quotation-marks screws things up, resulting into: /argo/staging/script: line 1: echo hello world: command not found

Obviously I can't run the function without the quotation marks as it isn't legal python code:

def hello():
    echo hello world

How could I resolve this?

[flaviuvadan]

Hi @Trollgeir 🙂 I believe you're looking for this example! A similar example that runs 2 bash commands with Hera:

"""This example showcases how to run a container, rather than a Python, function, as the payload of a task in Hera"""

from hera import Task, Workflow, InputParameter, OutputPathParameter, set_global_host, set_global_token

set_global_host(...)
set_global_token(...)

w = Workflow('container')

g = Task(
    'generate',
    image='alpine:latest',
    command=['sh', '-c'],
    args=["echo $RANDOM > result"],
    outputs=[OutputPathParameter('result', '/result')],
)
c = Task(
    'consume',
    image='alpine:latest',
    command=['sh', '-c'],
    args=["echo result was: {{inputs.parameters.message}}"],
    inputs=[InputParameter('message', g.name, 'result')],
)

g >> c
w.add_tasks(g, c)
w.create()

[Trollgeir]

Hi @Trollgeir slightly_smiling_face I believe you're looking for this example! A similar example that runs 2 bash commands with Hera:

"""This example showcases how to run a container, rather than a Python, function, as the payload of a task in Hera"""

from hera import Task, Workflow, InputParameter, OutputPathParameter, set_global_host, set_global_token

set_global_host(...)
set_global_token(...)

w = Workflow('container')

g = Task(
    'generate',
    image='alpine:latest',
    command=['sh', '-c'],
    args=["echo $RANDOM > result"],
    outputs=[OutputPathParameter('result', '/result')],
)
c = Task(
    'consume',
    image='alpine:latest',
    command=['sh', '-c'],
    args=["echo result was: {{inputs.parameters.message}}"],
    inputs=[InputParameter('message', g.name, 'result')],
)

g >> c
w.add_tasks(g, c)
w.create()

I looked at this, but this setup doesn't utilize a script-file, and is limited to short commands. I have several lines that I want to be executed.

Would a solution be to add a script_source argument which the user can fill in whatever string they want, which is only legal if no function was passed?

Edit: I played around a bit by adding script_source as an optional argument for a task, which I added (if present) directly to the source field in script_kwargs with success. This unlocks a lot more usability for custom implementations for the user, but I'm unsure if this is the best way to implement it. I could put up my current implementation in a PR (including some validations).

Thoughts?