Recurrence with 6 months recurs every year

[BrianChapman]

Expected Behavior

I expect the recurring task to repeat every 6 months.

Current behaviour

When I create a task to repeat every 6 months and mark it as complete it reschedules the next task 12 months out.

• [ ] Test recurring task 🔁 every 6 months ⏳ 2023-08-30
• [x] Test recurring task 🔁 every 6 months ⏳ 2022-08-30 ✅ 2022-08-30

If I schedule a task to repeat every 180 days, it does the right thing

• [ ] Test recurring task 🔁 every 180 days ⏳ 2023-02-26
• [x] Test recurring task 🔁 every 180 days ⏳ 2022-08-30 ✅ 2022-08-30

Steps to reproduce

Create a recurring task to repeat every 6 months like so:

• [ ] Test recurring task 🔁 every 6 months ⏳ 2023-08-30

then mark it as done

Which Operating Systems are you using?

• [ ] Android
• [ ] iPhone/iPad
• [ ] Linux
• [X] macOS
• [ ] Windows

Obsidian Version

0.15.9

Tasks Plugin Version

1.12.0

Checks

• [X] I have tried it with all other plugins disabled and the error still occurs

Possible solution

No response

[claremacrae]

What's going on

@BrianChapman, Thank you for the clear report.

Current behaviour

When I create a task to repeat every 6 months and mark it as complete it reschedules the next task 12 months out.

• [ ] Test recurring task 🔁 every 6 months ⏳ 2023-08-30
• [x] Test recurring task 🔁 every 6 months ⏳ 2022-08-30 ✅ 2022-08-30

The reason for this behaviour is that 6 months after 2022-08-30 is 2024-02-30 which does not exist, so the library that Tasks uses for managing dates skips forward the next increment until it finds a valid date.

There's more information about this in https://github.com/obsidian-tasks-group/obsidian-tasks/issues/614, which is an outstanding request to document these limitations.

If I schedule a task to repeat every 180 days, it does the right thing

• [ ] Test recurring task 🔁 every 180 days ⏳ 2023-02-26
• [x] Test recurring task 🔁 every 180 days ⏳ 2022-08-30 ✅ 2022-08-30

Yes, using a fixed increment such as a number of days is the only safe workaround for this.

Optional request for help

Apologies that you had to discover and write up a known limitation yourself. I'm struggling a bit with the sheer volume of stuff to do.

If you had the time and inclination, I've added a description of what would be involved in updating the docs to help others avoid this issue: https://github.com/obsidian-tasks-group/obsidian-tasks/issues/614#issuecomment-1232482650 - and a pull request would be very welcome indeed.

[claremacrae]

I removed the bug label from this, because it is the intended behaviour of the library we are using.

However, I do wonder whether there is anything that could be done in Tasks to warn users of the problem.

Things I can think of:

• Detect if the new date is unexpectedly larger than the recurrence interval.
• And if so, do 1 or more of:
  • Pop up a warning message perhaps linking to the docs and explaining that interval(s) were skipped...
  • Add an extra bullet point in the user's file, between the old and new tasks, with an explanation
    • The problem with this is that they won't see if it they are completing tasks via a Tasks query block

[BrianChapman]

Ok thanks for the explanation. I guess my search terms didn't turn up the existing bug. "every 6 months" can be interpreted different ways by different people depending on their intentions. I will look into adding a PR to clarify the behavior in the docs.

On Tue, Aug 30, 2022, 10:59 PM Clare Macrae @.***> wrote:

I removed the bug label from this, because it is the intended behaviour of the library we are using.

However, I do wonder whether there is anything that could be done in Tasks to warn users of the problem.

Things I can think of:

• Detect if the new date is unexpectedly larger than the recurrence interval.
• And if so, do 1 or more of:
  • Pop up a warning message perhaps linking to the docs and explaining that interval(s) were skipped...
  • Add an extra bullet point in the user's file, between the old and new tasks, with an explanation
    • The problem with this is that they won't see if it they are completing tasks via a Tasks query block

— Reply to this email directly, view it on GitHub https://github.com/obsidian-tasks-group/obsidian-tasks/issues/1087#issuecomment-1232497378, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACNXSMSVHUFIRPDDCY72L3V33YF3ANCNFSM6AAAAAAQA3MFIM . You are receiving this because you were mentioned.Message ID: @.***>

[claremacrae]

Feedback received on Discord:

I know it's a known bug, but it's not just a bug, this is a trust-breaker. How can I use Tasks if I must verify every completion manually? Are you for real? How hard is it to code something like "in 5 months = shift date 5 months forward, if there's no 29th feb then pick next possible day = 1st mar" or even better "in 5 months = get next 5 months durations in days, shift sum days", or just simple stupid "in 5 months = shift month by 5, if there's no such date, shift 5x30 days"? How is this not first priority? I fully expect answer "this is OSS, you don't like it you don't use it, you want it fixed you fix it yourself" but I just can't believe it. A date-management plugin can't manage dates.

using latest 1.14.0 Tasks

[dima-stefantsov]

This is a bit out-of-context, it's not that useful in here where we are clear the bug exists. It is also not cool that you are sharing my personal information such as linking my discord profile to my name and github profile.

My feedback is that

1. not being able to rely on my date management system is trust-breaking, and trust is the most important thing for productivity systems (according to me and David Allen too).

2. this bug is very severe. Mistaking a date by 6 months? No thanks!

3. this bug is known for half a year. And yet it's not mentioned in docs, it is so severe I'd expect it being on a front page! For example BrianChapman have encountered it. And there are many others who did too, but didn't find time to report it. And even more others who are just going to miss their planned actions never noticing the skipped months.

4. it is still not fixed, I've tested latest 1.14.0

Therefore I rate this bug as highest possible priority, and since proper fixes might be hard to implement, I propose some fixes that will be really easy to do, and yet Tasks users will not be screwed as much as they are:

• don't let us use "every month", only let us use "every 30 days"
• if next date doesn't exist, take the next day, not the next interval
• if next date doesn't exist, take previous day, 31st Jan -> 28th Feb -> 28th Mar
• if next date doesn't exist, convert months to x30 days
• if next date doesn't exist, calculate following months durations in days, shift sum days
• lookbehind info completed tasks log to find previous date state, to support 31->28->31

I could even provide some pseudocode:

$date_start = 2022-08-31
$recurrence_rule = every month
$next_due = rrule::get($date_start, $recurrence_rule) //2022-10-31
// i guess complication comes from somewhere here,
// you have absolutely no control and understanding about the $recurrence_rule,
// there's just magic happens in rrule lib, and it fails.
// 
// otherwise you'd have probably fixed it much earlier

since we are not trying to really solve it (you couldn't do it in 6 months I guess it may be hard?), let's try to work around it:

$date_start = 2022-08-31
$recurrence_rule = every month
$next_due = rrule::get($date_start, $recurrence_rule) //2022-10-31

// function append_on_the_last() checks for having a "month" and not having " on ",
// if true appends " on the last" to a rule.
$recurrence_rule_hotfix = append_on_the_last($recurrence_rule) // every month on the last
$next_due_hotfix = rrule::get($date_start, $recurrence_rule_hotfix) //2022-09-30

if ($next_due > $next_due_hotfix) then $next_due = $next_due_hotfix
// aka
$next_due = min($next_due, $next_due_hotfix)

return $next_due

for "when done" cases I expect $date_start to already be set to $today, append_on_the_last() function would insert before "when done" string part of the $recurrence_rule

This would be an implementation of my third proposal, 31st Jan -> 28th Feb -> 28th Mar. It just works. This fix would take about 30 minutes to implement fast, or maybe few hours to think through a good architecture like checking for "month" in recurrence rule before making calculations, making it beautiful etc.

This is programming. Nothing is ever hard here. Especially when we are talking about a+b=c simple math. How much you expect to be mistaken in such calculations - up to you. It's a bit fine to have a+b=c-1, it's not fine to have a+b=2c

wanna code for 4th proposed solution?

$date_start = 2022-08-31
$recurrence_rule = every month
if ($recurrence_rule contains "month" && $recurrence_rule !contains " on ") then
    $recurrence_rule = replace(/month(s)/30 days/) // find a number before "month" and multiply 30 by it
$next_due = rrule::get($date_start, $recurrence_rule) //2022-10-31

return $next_due

it's still better than missing whole extra month.

code for 1th proposed solution is especially beautiful

$recurrence_rule = every month
if ($recurrence_rule contains "month") then throw error "not supported, use 30 days instead"

it is still better than trusting the system to fulfill the date and then miss it. I can go on, you've got the point. I hope this was useful or at least amusing.

[dima-stefantsov]

Also found a new bug which would make my 3rd-proposal-implementation obsolete sadly.

throw this rrule away lol, it will eat us all )

[claremacrae]

It is also not cool that you are sharing my personal information such as linking my discord profile to my name and github profile.

OK, I have removed that. I'll reply in more detail in the next comment.

[claremacrae]

@dima-stefantsov,

Thank you for the really helpful comments and ideas in that comment above.

In summary, one of your ideas paved the way to a more complex algorithm that has enabled fixing the bug for all cases, which has been implemented by @schemar and will be released soon. See PR #1197.

---

The problem

Yes, you hit the nail on the head with your comments in this snippet:

$next_due = rrule::get($date_start, $recurrence_rule) //2022-10-31
// i guess complication comes from somewhere here,
// you have absolutely no control and understanding about the $recurrence_rule,
// there's just magic happens in rrule lib, and it fails.
// 
// otherwise you'd have probably fixed it much earlier

That was exactly the problem.

if next date doesn't exist, convert months to x30 days

Yes, all my previous ideas for fixing this had been around that kind of thing, but I was uncomfortable with them for a number of reasons, including that if it was, say, every 15 months, there would be quite a lot of drift away from the user's intention.

---

The fix

if next date doesn't exist, take previous day, 31st Jan -> 28th Feb -> 28th Mar

Thank you! That comment of yours was the key idea to finding a fix for the bug, which is now in progress in #1197.

This is the algorithm I came up with, some hours after reading that comment:

1. find an algorithm to work out if, after the incrementing, an interval had been skipped - both for months and for years.
2. instead of adjusting the due date in the new task, adjust the due date in the original task, moving it a day earlier and try repeating again
3. keep doing 2 in a loop, moving the due day earlier one day at a time, until it gives a new date in the correct month or year, that did not do any skipping. (Most dates, it only needs 1 iteration, but from 31st of any month to 28 Feb needs several)

When I shared that with @schemar he implemented it really quickly and really well.

In particular, I hadn't figured out an even remotely clean way to do step 1, for the many possible month and year recurrences. But schemar came up with a really clean idea for that.

So, all in all, a team effort. Thank you to you and to schemar for enabling this to be fixed.

---

What happens next?

There are some more scenarios I need to add tests for - in particular, to ensure that it can't ever get stuck in an infinite loop, and to make sure it works with other month and day patterns than the simple ones.

On the latter, some of your comments suggested a few more test cases that I need to add. Thank you.

(I find it's common to spend a lot more time thinking about tests for a feature or fix than doing the coding/)

Then it will be ready for release, which I will of course do as soon as I can.

[claremacrae]

An update on progress on this...

On further testing, I found that the following now gets stuck on the same date and never moves forward, instead of previously skipping February:

- [ ] #task do stuff 🔁 every month on the 31st 📅 2022-01-31
- [x] #task do stuff 🔁 every month on the 31st 📅 2022-01-31 ✅ 2022-10-03
- [x] #task do stuff 🔁 every month on the 31st 📅 2022-01-31 ✅ 2022-10-03
- [x] #task do stuff 🔁 every month on the 31st 📅 2022-01-31 ✅ 2022-10-03

It is not as trivial as adding special case code for on the, in part because rrule offers multiple different ways to write that.

If you want to track progress and research on this, or have any suggestions on how to fix it, please subscribe to #1197 and add any comments there.

[claremacrae]

Release of the fix for this is unfortunately blocked until the above issue is fixed.

[dgreen]

I just tried Apple Calendar and BusyCal on the iPhone, and they skip November when I schedule October 31st and monthly repeat. The monthly repeat in BusyCal is shown as Monthly on the 31st after I save the entry. I found this note https://ux.stackexchange.com/questions/36889/calendar-recurrences-on-the-last-day-of-the-month about Outlook and Google from 9 years ago, noting the challenges.

A trio of thoughts:

1. I wonder if there is covered by a standard like iCalendar.

2. I generally want either the last day of the month or the last business day of the month. I wonder if this could be added as an additional way to specify periodicity.

3. The fundamental challenge seems to be that the user's intention to be on the 31st is lost when moving to a non-31-day month for the following month. (Similar problem going across February, of course.)

[claremacrae]

For anyone interested in seeing the potential final fix for this, you can see the significant additions to the documentation here:

https://github.com/obsidian-tasks-group/obsidian-tasks/pull/1197/files#diff-7180dda3f3ee0ae61f9611e70893a879113dfb5d175cc4cce691ecc3b940ced4

And you can download the Tasks-Demo vault with the fix included, and a distribution of the new build for use in your own test vault, under the Artifacts section here:

https://github.com/obsidian-tasks-group/obsidian-tasks/actions/runs/3200900723

Please record any comments in https://github.com/obsidian-tasks-group/obsidian-tasks/pull/1197

[claremacrae]

This has been significantly improved in release 1.15.1.

Note that if you say every month on the 31st, it will still skip months. Use every month on the last instead.