step hooks

[chirag04]

trafficstars

Is there any thought being put into supporting something like a pre and post step hook for a workflow? eg: in case of a ci server, steps are user provided. as a workflow executor service, one may want to do some pre processing or post processing for every step.

[alexei-led]

For “pre” step you can use K8s standard approach- initContainers.

[sdminonne]

Not sure I'm understanding the issue here. Can't you just add a pre-processing step or a post-processing (or both) for each process requiring them?

[clamoriniere1A]

@chirag04 do you want to run your Pre/Post as others Jobs(Pods)? Or do you need to be in the same Pod than the Step?

[chirag04]

@alexei-led problem with initContainers is that there is no equivalent terminateContainer.

@sdminonne @clamoriniere1A ideally something that runs outside the context of the step(job/pod). We can add a pre-processing/post-processing step tho.

1. it would be awesome if we can support pre/post processing step(Job) in workflow-controller out of the box.
2. I was wondering if there is something light weight compared to pre/post processing job. there is a little bit of overhead in scheduling these new pre/post containers for every step.

[alexei-led]

@chirag04 Can you put everything inside initContainers, plus single container in containers? They run in sequence, thus the first container can serve as "pre-step" and one container in containers section will always run as last ("post-step") all other containers within initContainers section may contain actual step/s you want to launch. I understand that syntax wide it's a bit misleading, but will give you desired functionality.

[alexei-led]

@chirag04 another approach is to use K8s Container lifecycle events postStart and preStop, see here

...
containers:
  - name: lifecycle-demo-container
    image: nginx
    lifecycle:
      postStart:
        exec:
          command: ["/bin/sh", "-c", "echo Hello from the postStart handler > /usr/share/message"]
      preStop:
        exec:
          command: ["/usr/sbin/nginx","-s","quit"]

[chirag04]

@alexei-led Thanks for the note. I think the use case here is to execute some control logic before the step container command executes. postStart hook is too late for that. I like the idea of augmenting each step with a pre-processing/post-processing step that @sdminonne mentioned. It would be awesome if we can support such containers out-of-box here.