How to stop (and restart?) individual flows?

[hjoliver]

Following on from @dpmatthews 4 days ago

cylc stop currently supports --flow=INT which means "Stop flow number INT from spawning more tasks". I don't really like this for 2 reasons:

• If you stop a workflow can you restart it using play. This isn't true if you "stop" a flow.
• If you want to stop a flow I think it is more likely that you want to remove any tasks that belong to that flow from n=0 rather than simply preventing the spawning of more tasks.

Personally I think cylc stop --flow=INT is the natural way to stop a flow.

We can't just "remove any tasks that belong to that flow from n=0" if those (active-waiting) tasks also have other flow numbers. Removing the flow number stops that flow, but allows the other flows (if any) to continue.

We can however remove active-waiting tasks from n=0, before they run, if they don't belong to any other flows.

We should also consider allowing cylc play --flow=INT to restart a stopped flow (e.g. in case I stopped the wrong flow by mistake).

This, along with cylc stop --flow=INT, would not be inconsistent with cylc play releasing tasks in a running workflow as well as starting a new workflow from scratch.

(However, we'd have to think through what it means to restart a flow that was wholly or partly merged when it was stopped).

[oliver-sanders]

If we stop a flow by removing it's number how do we know which tasks to add the number back to in order to restart it?

Perhaps we would multiply the flow number by -1 and treat negatives as no-flow tasks?

[hjoliver]

I can imagine several ways, but that's not a bad idea. Another might be to have a special stopped flow store in the DB.

[hjoliver]

(Notes post project meeting)

• ability to stop a flow doesn't necessarily require the ability to restart it, because starting another flow at the right place would presumably achieve the same thing
• the current mechanism stops the flow continuing from wherever it is in the graph at the time the command is issued, which might sometimes be inconvenient (you might have to wait and watch for the right moment to stop it)
• instead, or as well, we might need the ability to:
  • specify a future stop point at the outset (i.e. at trigger time), e.g. don't continue beyond some cycle point, or don't go beyond some task or tasks (note this goes back to the SoD proposal, but hasn't been implemented yet)
  • specify a future stop point on the fly, perhaps using something like cylc set-outputs to make it look like the flow already ran those tasks

[oliver-sanders]

cylc stop one//1/foo --flow=2 is currently permitted but probably doesn't do what it says on the tin.

[hjoliver]

cylc stop one//1/foo --flow=2 is currently permitted but probably doesn't do what it says on the tin.

Definitely not intentional (if my fault, at least). Fix ... #4749

[wxtim]

One point which occurs on a quick read of this issue is a language problem: In this context I think "flow"=="reflow", but when using cylc install --flow-name=foo, "flow" is "workflow". I'm not sure we should depend on context to distinguish what "flow" is short for.

[oliver-sanders]

cylc install --flow-name=foo

Agreed, I think --flow-name should be renamed to --id or similar (might not be the full ID because of the --run-name but we can explain that in the CLI docs)

Bumping this to another issue: https://github.com/cylc/cylc-flow/issues/4786

[oliver-sanders]

Note, #5643 allows flow numbers to be stripped from tasks. Currently this does not include recording which numbers have been stripped to allow flows to be restarted.