calendarspec: add support for "every n days"

[claui]

Summary

Allow a period ("every n days") to be associated with a weekday spec.

For example, the calendar string:

Mon/14

will be normalized to:

Mon/14 *-*-* 00:00:00

and will trigger every two weeks.

Rationale

Prior discussions/RFEs: https://github.com/systemd/systemd/issues/6024 https://github.com/systemd/systemd/issues/11768

Design

I chose to re-use the existing repeat (/) modifier that the calendar spec already uses for dates and times. My hope is that it’s easier to memorize and less confusing compared to introducing yet another symbol.

For consistency, the multiplier is given in days, not weeks.
I felt that fits the existing semantics a little better, i. e. to match "the value plus all multiples of the repetition value."

In other words, the value is a weekday so I feel the repetition value should be measured in days, too. As an added benefit, it makes the calendar string more easily understood because at a glance, the meaning of Mon/14 should be more obvious than Mon/2.

Limitations

For now, the period must be a positive multiple of 7.
I’m sure others might feel that other values would be useful too. However, I couldn’t figure out a good syntax that would make sense in a calendar string. (For example, I find it difficult to convey that Mon/10 should be interpreted as every 10 days. So I left that invalid for now.)

Implementation note

I’ve introduced a new WeekdaySpec struct because I felt that the existing CalendarComponent struct didn’t quite fit.
But that’s just a design choice. I have no strong opinion about it.

[claui]

I’m having trouble figuring out what test-oomd-util is and how to reproduce its failure.

This is what happens when I run sudo build/test-oomd-util:

Error converting 'MemTotal:       32495256 kB trailing' from /oomdgetsysctxtestjCqCs0to uint64_t: Invalid argument
Last pgscan 33 greater than current pgscan 1 for /herp.slice/derp.scope. Using last pgscan of zero.
Last pgscan 33 greater than current pgscan 2 for /zupa.slice. Using last pgscan of zero.
Last pgscan 33 greater than current pgscan 2 for /zupa.slice. Using last pgscan of zero.
Last pgscan 33 greater than current pgscan 2 for /zupa.slice. Using last pgscan of zero.
Last pgscan 33 greater than current pgscan 2 for /zupa.slice. Using last pgscan of zero.
Last pgscan 33 greater than current pgscan 1 for /herp.slice/derp.scope. Using last pgscan of zero.
Last pgscan 33 greater than current pgscan 1 for /herp.slice/derp.scope. Using last pgscan of zero.
Bus n/a: changing state UNSET → OPENING
sd-bus: starting bus by connecting to /run/dbus/system_bus_socket...
Bus n/a: changing state OPENING → CLOSED
Failed to connect to system bus: No such file or directory
Couldn't allocate a scope unit for this test, proceeding without.
Found cgroup2 on /sys/fs/cgroup/unified, unified hierarchy for systemd controller
Failed to attach 20447 to compat systemd cgroup /: No such file or directory
test-oomd-util: cgroups are not running in unified mode, skipping tests.
test-oomd-util: system does not support pressure, skipping tests.

It then returns with an exit status of 0.

~~I’d appreciate a pointer on how to start investigating this.~~
Update: I’m seeing the same CI failure in several recent PRs so I assume it’s unrelated to the PR at hand.

[yuwata]

Yeah, the failure is #20655. Please ignore it.

[claui]

Rebased on latest main, now that #20655 appears to be fixed.

[poettering]

Hmm, I figure we should make */10 *-*-* 00:00 work too then?

[poettering]

Hmm, I figure we should make */10 *-*-* 00:00 work too then?

Hmm, or maybe not, because the "phase", i.e. the first day when to start the repeat cycle is unclear then.

[poettering]

So, hmm, I think I like the syntax, in the sense that xyz/r a-b-c d:e:f would mean: find first point in time xyz a-b-c holds, then trigger at that time, plus all r days later, until the last time xyz a-b-c holds.

[poettering]

Hmm, so I'd really like to discuss first how the syntax should be interpreted, i.e. what does this mean:

Mon,Tue/10 *-*-* 00:00 → ?
*/10 1-1-2021 00:00 → ?
Mon/10 1-1-2021 00:00 → ?
Mon/1,2,9 1-1-2021 00:00 → ?

I kinda have the feeling that it would be simplest to just declare that the /… suffix expression for the weekday should just be considered a repetition expression in days for the whole weekday+date part of the full expression, i.e. that if you have abc/r x-y-z as expression, that this means abc x-y-z is the "start date", and then the r just stupidly repeats based on that start date. I think that would make implementation simple and understanding things for users pretty easy too. However, implementing it like that would of course meaning any range expressions in the date part otherwise are kind pointless...

[claui]

Mon,Tue/10 *-*-* 00:00 → ?

Triggers on te first Tuesday after the beginning of the Unix epoch, and every 10 days after that. Additionally, triggers every Monday.

*/10 1-1-2021 00:00 → ?

Triggers on Friday, 2021-01-01 (and that day only).

Mon/10 1-1-2021 00:00 → ?

Never triggers, as 2021-01-01 was a Friday.

Mon/1,2,9 1-1-2021 00:00 → ?

Reject as EINVAL, as it’s probably not worth supporting.

i.e. that if you have abc/r x-y-z as expression, that this means abc x-y-z is the "start date", and then the r just stupidly repeats based on that start date.

That might work. One thing that bothers me a bit though is that Mon and Mon/7 would then mean different things. Not entirely sure yet whether that’s a flaw or a feature.

[poettering]

so if we go the way that the new addition just stupidly repeats events in some day interval after the first time it triggers, maybe it would actually make more sense to write it after the date, not the weekday. So, what about doing it with this syntax:

abc x-y-z//r d:e:f

i.e. use a double // as separator for a "repeat this every r days" expression placed after the date, and before the time. Example:

Tue 2021-9-14//9 7:05

[poettering]

so, here's another idea: there's an RFE somewhere to extend the calendar logic in .timer units with additional settings "NotBefore=" and "NotAfter=" which take explicit timestamp specifications (i.e. not repeating calendar expressions), and would define the absolute time range we'd apply the timer patterns to. If we had that, then I think it would be pretty natural to say that any such day repeat logic could mean "every nth day after the NotBefore=" date, and then we'd match in addition to the actual date expression.

So what about this:

NotBefore=2021-01-01
OnCalendar=Mon *-*-*//4

This combination would then mean: trigger on the every monday that is a multiple of 4 days past Jan 1st 2021. Specifically, this would then be the series 2021-01-25, 2021-02-22, …

Or in other words, the NotBefore= would give us the "anchor" for the repetition, so that we then don't have to to derive the anchor from the date expression itself anymore.

And of course, if NotBefore= is not specified we'd use the UNIX epoch for that. I think this would be really straightforward to use for people: they'd just configure the match expression plus the start date, and then a repetition interval and all would be good.

[claui]

so, here's another idea: there's an RFE somewhere to extend the calendar logic in .timer units with additional settings "NotBefore=" and "NotAfter="

(Note to self: #16733)

Or in other words, the NotBefore= would give us the "anchor" for the repetition, so that we then don't have to to derive the anchor from the date expression itself anymore.

I feel we’re onto something workable here. Going to map out a couple more tests to (hopefully) discover the relevant edge cases.

[keszybz]

Setting the "needs rework" label based on the latest comments.

[poettering]

any update?

[claui]

@poettering Not at this time. Planning to tackle it in April when I have a couple of days off.

[poettering]

Ping?

[yuwata]

Is this still draft?

[claui]

@yuwata Yes, this is still a draft.

The main pending task is to take into account most of @poettering’s feedback from these comments: https://github.com/systemd/systemd/pull/20711#issuecomment-918543848, https://github.com/systemd/systemd/pull/20711#issuecomment-918911656, https://github.com/systemd/systemd/pull/20711#issuecomment-918932975.

[claui]

@poettering Just rebased, now tackling the idea from #16733 and thinking about suitable examples.

[poettering]

ping?

[poettering]

ping?

[Robert-Riedl]

any update on this @claui ?

[WarpinWolf]

Hi there, really appreciate your hard work. Nevertheless - would like to ask whether there are any news here to share.

I would also like to mention https://github.com/systemd/systemd/issues/6024 - after reading it I personally find that proposal much more intuitive than what has been discussed here with "//" (the "/" already gave me a hard time to understand it).

Thanks again Wolf