CPU and Max RSS Analysis tools

[ChrisPaulBennett]

This apart of 3 pull requests for adding CPU time and Max RSS analysis to the Cylc UI.

This adds the Max RSS and CPU time (as measured by cgroups) to the table view, box plot and time series views.

Linked to; https://github.com/cylc/cylc-flow/pull/6663 https://github.com/cylc/cylc-uiserver/pull/675

Check List

• [x] I have read CONTRIBUTING.md and added my name as a Code Contributor.
• [x] Contains logically grouped changes (else tidy your branch by rebase).
• [x] Does not contain off-topic changes (use other PRs for other changes).
• [x] Applied any dependency changes to both setup.cfg (and conda-environment.yml if present).
• [x] Tests are included (or explain why tests are not needed).
• [x] Changelog entry included if this is a change that can affect users
• [x] Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/XXXX.
• [x] If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

[MetRonnie]

Btw it looks like you have accidentally pushed this branch both to your fork and this upstream repo. I'm deleting the branch on here but it will remain on your fork.

[MetRonnie]

We'll get round to this when we can, but a little bit swamped at the moment!

[wxtim]

I've had a first look -

The "chips" look good:

• Is there a reason I don't get maxrss for the hpc background job?
• Might these be more useful if the data was converted to human readable?

If nothing has finished the analysis view sticks in this state:

I threw a fairly nasty workflow at it and noticed some anomalies:

Cannot page through, cannot see tasks I know to have completed.

[ChrisPaulBennett]

Oliver recommended the best way to move data around cylc was to use a debug message. Those chips only only appear in dev mode and won't be visible to normal users. What's a HPC background job? The analysis view being stuck in that state was introduced when I merged in @oliver-sanders changes to SQL. I haven't figured out what is causing this yet. Clicking refresh will solve the issue.

[wxtim]

What's a HPC background job?

Using the supercomputer login node as just another computer. We use the term background to mean execution a job in bash without using a job runner like PBS or Slurm.

E.g. > ssh exab bash program.sh rather than ssh exab qsub program.sh

[!WARNING] Irrelevant @oliver-sanders pointed out to me after I posted the original comment that background jobs don't use cgroups in the same way, so the profiler probably won't work for them anyway.

[oliver-sanders]

Is there a reason I don't get maxrss for the hpc background job?

1. You would need to configure this platform to run the profiler.
2. There are no cgroups to poll for background jobs so it wouldn't work if you tried.

Those chips only only appear in dev mode and won't be visible to normal users.

Debug is def the right level to send these messages at, however, I'm not sure we've got the UI to hide debug level messages yet. Orthogonal to this PR either way.

[oliver-sanders]

Welcome to the project @samuel-denton :wave:.

Please could you add your name to the list here: https://github.com/cylc/cylc-ui/blob/master/CONTRIBUTING.md#code-contributors.

(we have to do this with the first commit to each repository for legal reasons)

Note: If your GH display name differs from the format provided, you'll need to add a .mailmap entry (ask us if needed).