cylc-flow icon indicating copy to clipboard operation
cylc-flow copied to clipboard

Should cylc config be able to report the configuration being used by a running workflow?

Open dpmatthews opened this issue 2 years ago • 6 comments

Given a workflow id, cylc config prints the "workflow configuration" (according to the help text). But what does that really mean? What (I think) it means is that it parses whatever configuration files it finds in the workflow directory. However, this may be very different from the configuration in use by a running workflow because: a) the config may have been changed by reinstall but not reloaded b) the config may have been altered via cylc play options (--set #4740, --fcp, etc)

The parsed config file is recorded in the log/flow-config directory. This includes the result of any --set options although note that it still doesn't reflect any changes made by --fcp for example. (See also #4755)

The Rose fcm make built-in app uses cylc config to check for the existence of "fcm_make2" tasks. In theory it could get the wrong result because it is not actually querying the configuration in use. In practise this is very unlikely and not worth worrying about for this legacy use case. However, it does raise the question: do we need the ability to query the configuration being used by a running workflow?

dpmatthews avatar Mar 23 '22 11:03 dpmatthews

However, it does raise the question: do we need the ability to query the configuration being used by a running workflow?

Good point. I think we should, ideally. However, we could have another file in the run-directory that records the config of the (latest) running scheduler, rather than literally query the scheduler.

hjoliver avatar Mar 28 '22 08:03 hjoliver

  • Broadcasts alter the config in memory but not on disk.
  • We could allow the workflow config to be queried via GraphQL.
  • We need to be able to query the [runtime] section for #3751.
  • Does the fcm_make2 issue require [scheduling] information or is [runtime] sufficient?

oliver-sanders avatar May 17 '22 15:05 oliver-sanders

Does the fcm_make2 issue require [scheduling] information or is [runtime] sufficient?

I think [runtime] is sufficient but I'm really not worried about this use case.

dpmatthews avatar May 17 '22 16:05 dpmatthews

I don't think cylc config needs to be able to display the running config (note we get a file in log/config on reload), but we might want to open up an interface via GraphQL for other purposes.

Can we bump this to 8.x?

oliver-sanders avatar May 17 '22 16:05 oliver-sanders

Broadcasts alter the config in memory but not on disk.

We could potentially write every config change to disk, for provenance reasons. Short of that though, getting current active config via GraphQL would be fine IMO.

hjoliver avatar May 17 '22 23:05 hjoliver

Can we bump this to 8.x?

I don't this is super urgent, I'll bump it.

hjoliver avatar May 17 '22 23:05 hjoliver