cylc
data store: n=1 tasks should be created with the appropriate runahead status
Tasks which are beyond the runahead limit are only marked as runahead limited when they are spawned.
Here's a screenshot showing a cycle which is beyond the runahead limit:
The OS_START tasks show as runahead-limited, whereas the others don't. This is because the OS_START tasks are parentless i.e. n=0 so have been spawned, the other tasks are downstreams of OS_START so are n>0 tasks generated by the data store.
This is a cause of user confusion as it makes it look like the runahead limit has not taken effect. As per #3748 user's are not expected to understand why a task does or does not exist in the pool. Note it is possible that the GUI may show a runahead limited cycle with no runahead limited tasks (e.g. if viewing the n=2 window).
So IMO the data store should really be setting (and updating) the is_runahead flag so that all tasks beyond the runahead limit are displayed as limited irrespective of internal implementation details (i.e. spawning).
Please revert this PR when resolving: https://github.com/cylc/cylc-doc/pull/831
I'm not sure about this.
Technically, runahead limited means ready to run, but held back by the runahead limit.
So with foo => bar, if foo is runahead limited, bar is not - it is not being held back by the runahead limit, because it is not actually ready to run yet.
Note it is possible that the GUI may show a runahead limited cycle with no runahead limited tasks (e.g. if viewing the n=2 window).
That cycle is in the future graph, and is not being held back by the runahead limit because the scheduler has not even reached that part of the graph yet.
This is a useful distinction when you can't see the dependencies. If the runahead limit moves on, you can see that foo will start running immediately, but bar will not.
Should all future tasks be shown as queue limited too? (And xtriggers too, once we can show them at all).
As per https://github.com/cylc/cylc-flow/issues/3748 user's are not expected to understand why a task does or does not exist in the pool.
With the tiny exception that n=0 is special. That isn't just a matter of internal implementation, because the idea of an "active window" on the graph is important to understand.
I do agree with you in the case of parentless tasks. The runahead limit is the only reason (in principle, not just task pool implementation) that all future parentless tasks don't get submitted immediately.
Technically, runahead limited means ready to run, but held back by the runahead limit.
I thought you might say that!
IMO the "technically" bit is really an internal implementation detail, i.e. "that happens to be how it is presently implemented", rather than "that is what the concept conveys". Moreover, it's debatable since the concept only exists in the pool so the meaning for n>0 tasks is undefined.
From a behaviour perspective, the runahead limit defines a line in the sand past which the scheduler will not spawn.
1/a
2/a
3/a
-- P3 limit
4/a - wont happen until cycle 1 closes
5/a - wont happen until cycle 2 closes
6/a - wont happen until cycle 3 closes
The critical piece of information users need to be aware of is what side of this line a task is on. This might make the difference between an intervention being required or not. Consider an operator looking at a critical task in the tree view.
I don't think the distinction between "runahead limited" (i.e. parentless task) and "limited by a runahead limited task" (e.g. an n=1 task) is a useful one, so I favour the Cylc 7 behaviour of marking all tasks over the line as limited.
IMO the "technically" bit is really an internal implementation detail,
Technically, that was an unfortunate use of the word "technical" in the colloquial sense, by me!
I guess disagreement here boils down to this: should we put a runahead icon on:
- every task that is "beyond the runahead limit" OR
- every task that is "actively runahead limited" (i.e., it would run immediately if not for the limit).
(Note 2. is not just parentless tasks; it's any task with satisfied prerequisites as well).
Both of these are conceptual, not just technical implementation.
IMO the distinction does convey useful information. We often want to know why a particular waiting task is not running yet. With 2., we can see exactly which tasks have no unsatisfied prerequisites and will run as soon as the runahead limit advances.
This should not cause user confusion so long as they understand what the runahead limit is, and specifically that it is cycle point based.
Maybe we can have the best of both worlds:
- mark tasks that are actively runahead limited
- including all visible parentless tasks, which we are not doing now
- mark cycle points that are beyond the runahead limit
- this could be the runahead icon, and/or colouring (or greying out) the cycle point, and/or displaying a literal "line in the sand" across the display (possible in the tree view, and the table view when sorted by point,...)
We often want to know why a particular waiting task is not running yet
This used to be covered by the "show prerequisites and outputs" dialogue, a functionality which will be delivered in the new "metadata view".
IMO the distinction does convey useful information
We don't visually distinguish a waiting task with satisfied prereqs from a waiting task with unsatisfied prereqs, but in those cases the task will change state imminently (xtriggers aside) so point taken on the runahead limit 'ed side.
IMO it's more important to convey the runahead limit as opposed to tasks which are runahead limit 'ed so I'm still leaning towards (1).
This should not cause user confusion so long as they understand what the runahead limit is, and specifically that it is cycle point based.
I opened this issue because it did cause confusion, both with the user and myself. Taking a look at the above screenshot, it's not really clear that all of those tasks are behind the runahead limit. The user had also filtered tasks so the runahead limited ones weren't showing which made it even more confusing. The cycle point should definitely be marked as runahead as you suggested, but that still doesn't make it especially obvious and doesn't cover all bases, consider the table and graph views where the cycle point doesn't show and there is absolutely no information anywhere telling you that these tasks are behind the runahead limit.
The user had also filtered tasks so the runahead limited ones weren't showing which made it even more confusing
That's a good point.
consider the table and graph views where the cycle point doesn't show and there is absolutely no information anywhere telling you that these tasks are behind the runahead limit.
Cycle point does appear in the table view, but I guess sorting on other columns would complicate the runahead display problem a bit.
This used to be covered by the "show prerequisites and outputs" dialogue, a functionality which will be delivered in the new "metadata view".
Yes, but for this particular problem we can easily convey the relevant information without needing an additional click menu operation.
OK, how about this:
- mark runahead cycle points
- mark all tasks that are beyond the runahead limit, with a badge
- highlight the actively limited tasks (i.e., the only ones currently marked) by emphasizing the badge somehow (colour or contrast...)
Cycle point does appear in the table view
Only as a text string, the <cycle>/root node does not exist in the table view as it does in the tree view so does not have an associated task icon to convey the cycles runahead limit status.
mark runahead cycle points
:+1: (irrespective of the outcome of this issue, the current behaviour is not consistent with the handling of other states)
mark all tasks that are beyond the runahead limit, with a badge highlight the actively limited tasks
We could use a different symbol, but, would we want to complicate things just for this information?
I'm still not seeing the importance of being able to differentiate between a task which is exclusively runahead limited from one which is also limited by other dependencies, either way the task is not going to run until the runahead window allows. There are multiple types of dependency in Cylc (task outputs, inter-workflow triggers, xtriggers, ext-triggers, runahead limits, queues) all of which can hold a task back from running, we don't do anything special for other types of dependency, why do something special just to denote a particular combination of runahead limit and task output dependency states?
This distinction isn't even consistent when xtriggers are involved, in these cases tasks may not by exclusively runahead limited even when marked as such.
Note, historically, if users wanted to know why a task was held back, they used the "show prereqs and outputs" button. In the new GUI, the "metadata view" will provide this information along with task [meta], broadcast info and other useful things. IMO this is how users should differentiate between a task which is beyond the runahead limit and a task which is held back by the runahead limit alone. The metadata view is steadily climbing up the priority ladder, especially the prereqs/outputs part which is currently a glaring omission in Cylc 8.
mark runahead cycle points
Note, this is a client rather than a scheduler thing, Tui already marks a cycle as runahead if any task within that cycle displayed in the n-window is runahead. This means that the cycle will not be marked as runahead if all tasks within it are held back by at least one dependency.
This issue is causing a lot of confusion amongst our user base, most of whom are not used to seeing runahead tasks at all (they were filtered out by default at Cylc 7) and think that Cylc is ignoring the runahead limit they've configured.
OK (as per Element chat thread) - I concede defeat!
Personally I still think it's useful to distinguish:
- tasks that are ready to run as soon as the runahead limit moves on (i.e., actively limited)
- tasks that are downstream of those tasks and therefore have unmet prerequisites as well (i.e., the entire future graph)
However, I accept that marking all such tasks as runahead limited would not be incorrect and it probably would be easier to understand for most users.
Personally I still think it's useful to distinguish
Ditto, users need to know why a task is not being run, there are number of things that a task could be held back by:
- Task dependencies
- Xtriggers
- Ext-triggers
- Inter-workflow triggers
- Queues
- Held state
- Workflow paused state
- Runahead Limit
Having a separate icon to convey when any of these is the sole limiting factor is getting complicated, but we should look into other ways to convey this information prominently. E.G. we could have a task status message field (similar to the workflow status message) which could be displayed in the task context-menu, the table and possibly the tree views. It might read, "task held back by runahead limit" or "task held back by queue". Linking this into that issue.
For a clear(er) description of the current situation, at least, see https://github.com/cylc/cylc-doc/pull/824#discussion_r2083736536
I'm gonna throw in a different take:
Users should be able to tell which tasks are active/n=0 and which are inactive/window/n>0, at a glance..
I'm not sure what the solution would be.. Maybe if they're slightly faded and/or have an italic look .. Or some other solution. Regardless of the solution, this would make it clear that these tasks are not like the others.. Not only for the runahead case, but also for when users are trying to take actions on n>0 tasks that are not applicable (i.e. remove)
I'm forever telling my colleague to set n=0 to find out what tasks are actually holding up a workflow (as he likes to set n=3 a lot), usually these are old tasks that are waiting on an xtrigger that is not satisfied for some upstream reason..
@dwsutherland I agree that users should be able to distinguish n=0, i.e. "active" tasks, within the n-window without having to select 0-depth only, although that's really a separate issue from how runahead limiting is expressed in the GUI. I'm sure we must have discussed this before, but didn't find it in a quick look through cylc-ui issues - @cylc/core thoughts?
Well, I'm not sure it is an entirely separate issue; it means users can clearly see what tasks matter WRT being stuck/halted runahead or otherwise. And it doesn't make a lot of sense for clearly >n tasks (if clearly) to be held up, other than to say "these would be runahead blocked if active"
Only displaying the runahead icon for n=0 tasks is not a solution to highlighting the n=0 window (as it only serves as an indicator in niche circumstances) so this conversation is off topic for the current issue.
Yes, we need to distinguish n=0 regardless of runahead limiting, so it's a separate issue. (Even if not entirely separate as @dwsutherland says).
But, nevertheless, it came up here, so I just asked if anyone was aware of existing (esp. cylc-ui) issues that cover that- are you aware of any? If not, I'll post a new one.
Yes, we need to distinguish n=0 regardless of runahead limiting ... I'll post a new [issue].
... https://github.com/cylc/cylc-ui/issues/2179
Only displaying the runahead icon for n=0 tasks is not a solution to highlighting the n=0 window (as it only serves as an indicator in niche circumstances) so this conversation is off topic for the current issue.
Ok, let me state it in less general and on topic way:
n=1 tasks don't need to be marked as runahead if it's visually clear that they are inactive.
Because only thing it tells you is "these may be runahead blocked if active", but otherwise meaningless.
However I realize you could probably spin it another way, but my main point is active/inactive should be the distinguishing factor... (especially as, in many ways, you can't treat inactive tasks the same as active, WRT commands/actions)
Yes, however, a task is ahead/behind of the runahead limit whether it is n=0 or not. Only displaying this info if the task is "actively" runahead limited is very confusing for users.
I was initially of your opinion @dwsutherland but I've pretty much come around to @oliver-sanders perspective on this.
I updated the User Guide to explain the status quo better for 8.5, and this came up on the forum the other day too: https://cylc.discourse.group/t/wui-runahead-limit-is-showing-for-the-next-cycle-but-not-the-ones-afterwards/1175/2 ... but as hinted at the bottom of my post, I now think users will find it more useful just to see which cycles are below the runahead limit and which are beyond it.
(Along with active vs future).
(For a complete understanding of the system, which occasionally helps figure out what's going on in edge cases, it would be good if users understood the difference between "actively runahead limited n=0 task" and "n>0 task beyond the runahead limit", but the reality is that's way beyond most users).
Completely unrelated to this, but worth noting, we used to filter out runahead tasks in the GUI by default.
TBH, this is probably still a sensible default.
Bearing in mind, the Cylc 7 GUI only showed "n=0" (albeit for a much fatter task pool).
In Cylc 8, filtering out everything beyond the runahead limit might be too restrictive if you want to see the structure of the future graph.
However, it's definitely worth considering, as a default at least.
And for non-graph views, maybe it's pointless to show all those future tasks.