cylc vip

[wxtim]

Replaces #5094

#3896 - Implements the validate install play (VIP) working practice. I think that it works in principle for the other working practices.

Check List

• [x] I have read CONTRIBUTING.md and added my name as a Code Contributor.
• [x] Contains logically grouped changes (else tidy your branch by rebase).
• [x] Does not contain off-topic changes (use other PRs for other changes).
• [x] Applied any dependency changes to both setup.cfg and conda-environment.yml.
• [x] Tests are included (or explain why tests are not needed).
• [x] CHANGES.md entry included if this is a change that can affect users
• [x] Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/533.

[wxtim]

There a quite a few of these kicking about, worth running through the lot to make sure.

The text version of what your highlighting? Not the operator example? (I found plenty of the former, but none of the latter (I also found a string which had become accidentally truncated)).

[oliver-sanders]

Added a reftest in https://github.com/wxtim/cylc/pull/47

FYI: It's super easy:

1. Run the workflow with --reference-log, Cylc will generate a reference log in the run dir.
2. Copy the log file alongside the flow.cylc in the test dir.
3. Add --reference-test to the cylc play command.

[oliver-sanders]

Functional tests still failing :(

[wxtim]

Functional tests still failing :(

Looks like it was a rebase error.

[oliver-sanders]

Testing going well so far. I've spotted one oddity in the logged command:

$ cylc vip generic --workflow-name foo --no-run-name
$ cylc validate ~/cylc-src/generic
Valid for cylc-8.1.0.dev
$ cylc install .
INSTALLED foo from ~/cylc-src/generic
$ cylc play foo

The logged output says cylc install . which is incorrect as PWD=$HOME at the time.

[oliver-sanders]

The install part of vip seems to be skipping the previous-run check.

If I install the workflow directly...

$ cylc install generic --workflow-name foo
INSTALLED foo/run3 from ~/cylc-src/generic
NOTE: 2 runs of "foo" are already active:
  ▶ foo/run1 vld601.cmpd1.metoffice.gov.uk:43118
  ▶ foo/run2 vld601.cmpd1.metoffice.gov.uk:43002
You can stop them all with:         
  cylc stop 'foo/*'                 
See "cylc stop --help" for options. 

... any previous runs are listed. Whereas cylc vip doesn't display this:

$ cylc vip generic --workflow-name foo                                               
$ cylc validate ~/cylc-src/generic
Valid for cylc-8.1.0.dev
$ cylc install .
INSTALLED foo/run2 from ~/cylc-src/generic
$ cylc play foo
 
 ▪ ■  Cylc Workflow Engine 8.1.0.dev
 ██   Copyright (C) 2008-2022 NIWA
▝▘    & British Crown (Met Office) & Contributors
 
INFO - Extracting job.sh to ~/cylc-run/foo/run2/.service/etc/job.sh
foo/run2: vld601.cmpd1.metoffice.gov.uk PID=128496

This also means the --no-ping option is currently functionless in cylc vip.

Should try to get the previous-run-scanning working in cylc vip as the compound command makes it much easier to end up with multiple runs running in parallel.

[wxtim]

Should try to get the previous-run-scanning working in cylc vip as the compound command makes it much easier to end up with multiple runs running in parallel.

I think this is straightforward - I was looking at that code yesterday.

[wxtim]

The OptionSettings approach is looking much cleaner / more intuitive for people adding new args +1.

I think the kwarg constants (e.g. APPEND, HELP & DEST) can be removed now?

DEST is indeed no-longer required. The other two are still used. I've been able to remove ACTION, DEFAULT, METAVAR, CHOICES, OPTS, USEIF

[oliver-sanders]

All looking good to me, will finish off tomorrow.

[wxtim]

@datamel I found a new bug this very morning - would you care to check https://github.com/cylc/cylc-flow/pull/5121/commits/a22d9787d4bf9495d80abcb2850639340ff07840 too?

Did your testing include

cd ~/cylc-src/myworkflow
cylc vip

[datamel]

@datamel I found a new bug this very morning - would you care to check a22d978 too?

Did your testing include

cd ~/cylc-src/myworkflow
cylc vip

For info. This was tested by me but my setup uses localhost as the run host so it worked without error for me! Retesting this change now.