graphql: add [runtime] information to schema

[oliver-sanders]

We should expose the [runtime] section of the workflow configuration via the GraphQL schema.

This is needed for:

• Trigger edit (#3751).
• Displaying [runtime] in the "info" view (https://github.com/cylc/cylc-ui/issues/1071)
• Extraction of task configuration for workflow analysis.

This should be exposed for one-off queries but we don't need to (and probably wouldn't want to) expose this for subscriptions using deltas as that would involve maintaining the entire config in the data store.

Technical barriers:

• The config values need to be serialised:
  • (I.E. The configurations should be exposed as they were written in the flow.cylc file).
  • Parsec "coerces" values from their written format into the desired types (e.g. PT6H is converted into Duration(hours=6)).
  • We would need to "uncoerse" (i.e. serialise) these values back to strings. Parsec does not have an interface for this :(
  • Note we have to do this because GraphQL/JSON do not support things like isodatetime types.
• Broadcasts should be applied:
  • For trigger edit (and likely other uses) we would need to extract the [runtime] section for a given task instance (i.e. TaskProxy) rather than an abstract task (i.e. TaskDef).
  • This means we need to apply broadcasts to the [runtime] section before dumping it.
• We will need to decide what format to expose the [runtime] section in:
  • I.E. expose it as a multiline string or as a nested dictionary.
  • See also https://github.com/cylc/cylc-flow/issues/3751#issuecomment-789749725.
• Future tasks / historical jobs:
  • The main use cases here are for querying the [runtime] config of tasks which have not been run yet (i.e. future tasks).
  • Long term it would be good to query historical jobs.
  • E.G. I've trigger edited a task a couple of times, can I go back and see what config they ran with?

Extension:

• Add [runtime] config to cylc show?

Pull requests welcome!

[dwsutherland]

We have the following in task (definition) node:

Jobs have all the script information they're submitted with:

Do you want the scripting, environment (..etc) added to the task/family node? It's static, so I assume there (and not taskProxy)

Would make other configurations available via the GraphQL interface which may be useful elsewhere

I assume so from that point.. Because it might limit it if just left in the job node, served up, and sent back (for trigger-edit)

[oliver-sanders]

Ah right, we're already further along than I had expected :+1:. I'll try and have a poke at this.

We need enough information to reconstruct the [runtime] section of a task as it would appear in the flow.cylc file. E.G. for a "trigger edit" we need to be able to construct the [runtime] config in the UI so the user can edit it. So we need all of the [runtime] fields (if not done already) and we need to apply inheritance & broadcasts to them (if not done already).

Trigger edit example for this config:

# flow.cylc
[runtime]
    [[A]]
        [[[environment]]]
             x = x
    [[a]]
        inherit = A
        [[[environment]]]
            y = y
            z = z

If I applied this broadcast:

cylc broadcast -n a -s '[environment]z=42'

The UI would need to be able to construct the following file contents for the task ?/a:

[runtime]
    [[a]]
       inherit = A
       [[[environment]]]
           x =x  # inherited
           y =y
           z = 42  # previously broadcasted

So the user could edit it:

-            y = y
+           y = foo

The UI would then broadcast this diff back to the workflow and re-trigger the task.

[dwsutherland]

So we need all of the [runtime] fields (if not done already) and we need to apply inheritance & broadcasts to them (if not done already).

Broadcast, as I understand it (glob or not), is cycle specific .. So we'd probably need the [runtime] both at the definition level task/family and the proxy level taskProxy/familyProxy, with the latter the broadcast modified version that would show up in a trigger-edit. This way you could do a diff of base definition and any modifications (via broadcast/trigger-edit .. etc), and present this to the user.

If you're happy with this, I'll get started.. It will introduce broadcast deltas to the taskProxy/familyProxy node(s).

(will probably generalise this into a runtime/runtimeConfig field that can be used by task/family, taskProxy/familyProxy, and job (to replace the above/current, so we can view what settings a job ran with).