cycling: reinstate support for truncated datetime inter-cycle offsets

[oliver-sanders]

Truncated inter-cycle offsets used to work, but were broken, likely in the early releases of Cylc 8.

Description

This will validate fine under Cylc 7, but fail under Cylc 8:

[[[P1D]]]
    graph = """
        # "foo" from the 1st of the previous month
        foo[01T00-P1M] => bar
    """

The syntax is handy in situations where the inter-cycle offset is not constant.

Reproducible Example

(suite.rc)

[scheduling]
  initial cycle point = 2000
  [[dependencies]]
    [[[P1M]]]
      graph = extract => process
    [[[P1D ! 01T00]]]
      graph = extract[01T00-P1M] => process

[runtime]
  [[extract]]
    script = false

History

A test case was added that remains in the code to this day:

https://github.com/cylc/cylc-flow/blob/f413ebb919e9e85f773971d75a1ba8767afb4c20/tests/functional/cyclers/monthly_complex/flow.cylc#L9

However, no test file was added to run the test case, so the feature was never covered.

It would appear the feature was never documented either, however, that didn't stop it ending up in a workflow (likely via word of mouth) and being reported as a Cylc 7->8 migration issue!

[oliver-sanders]

It looks like 01T00 is erroneously identified as an absolute offset, but it's actually a truncated datetime i.e. relative to the cycle point.

This misidentification goes back to Cylc 7, however, correcting this does actually allow this example to run:

diff --git a/cylc/flow/task_trigger.py b/cylc/flow/task_trigger.py
index 857ad8276..118be111e 100644
--- a/cylc/flow/task_trigger.py
+++ b/cylc/flow/task_trigger.py
@@ -71,13 +71,24 @@ class TaskTrigger:
         self.offset_is_from_icp = offset_is_from_icp
         self.offset_is_absolute = offset_is_absolute
         self.initial_point = initial_point
-        # NEED TO DISTINGUISH BETWEEN ABSOLUTE OFFSETS
-        #   2000, 20000101T0600Z, 2000-01-01T06:00+00:00, ...
-        # AND NON-ABSOLUTE IRREGULAR:
+
+        # NOTE: Distinguish between absolute offsets
+        #   2000, 20000101T0600Z, 2000-01-01T06:00+00:00, 01T00, ...
+        # And non-absolute irregular:
         #   -PT6H+P1D, T00, ...
-        if self.offset_is_irregular and any(
-            self.cycle_point_offset.startswith(c)  # type: ignore[union-attr]
-            for c in ['P', '+', '-', 'T']
+        if (
+            self.offset_is_irregular
+            and self.cycle_point_offset
+            and (
+                # e.g: 01T00
+                self.cycle_point_offset[0].isdigit()
+
+                # e.g.: P1M, +0000000101, T00
+                or any(
+                    self.cycle_point_offset.startswith(c)
+                    for c in ['P', '+', '-', 'T']
+                )
+            )
         ):
             self.offset_is_absolute = False
 

However, it causes other issues as it causes foo[1] to be misidentified as not being an absolute offset.

It seems like it is impossible to tell whether an intercycle offset is absolute or not. For example is foo[0102]:

• An absolute offset referring to 01020101T0000Z.
• A relative offset referring to CCYY0102T0000Z.

It could be either!

[MetRonnie]

Another possibility could be to plug in the existing next()/previous() functionality which can currently only be used in the initial cycle point setting. E.g.

[scheduling]
  [[graph]]
    # 1st of the month:
    01T00 = extract
    # Every day:
    P1D = extract[previous(01T00)] => process

IMO this would be more readable than reinstating the truncated syntax, which looks like a sequence (i.e. every 1st of the month) and confusingly requires subtracting 1 month from the truncated point in order to get the current month (extract[01T00-P1M])

---

P.S. Cylc 7 graph for this example. The process task on the first of the month depends on the previous month's extract task, which is consistent with my proposed syntax.