wall clock timing functionality

[njsmith]

This is probably fairly low priority, but: an interesting and sometimes-useful feature is the ability to sleep until a particular wall clock time (e.g., "this cert is expiring at 2017-12-31T12:00:00Z, so I want to wake up 3 days before that so I can renew it"). This is quite different from trio's current timekeeping abilities, which are all oriented around monotonic time (which ignores clock changes, and stops while the computer is suspended, etc.). Really these are just different incommensurable time scales.

Fortunately, I don't think we need the ability to directly set a cancel scope deadline to a particular wall clock time, so this doesn't need to be integrated deep into the guts of trio's run loop. I think it'd be sufficient to provide a sleep_until_wall_clock_time (and maybe current_wall_clock_time for completeness).

This comment and the replies have some more details on this idea, including notes on how it could be implemented on different systems.

[njsmith]

The discussion in #394 got me thinking about this a bit again.

Contrary to the discussion in #168, it looks like on MacOS the best way to do this is not dispatch_after (which provides no way to cancel the submitted job), but rather to use dispatch_source_create ourselves (which is sort of like a timerfd, except that instead of a notification fd, it submits a job to a GCD queue when the timer fires – but we can reconfigure the timer at any moment). See documentation here. Also there are some global queues that are automatically run in the background that we can use, presumably dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0). (The "main" queue is not as relevant as it might sound -- main doesn't mean "this is the main thing to use", it means "this is for running things in the main thread", so if you want to use it then your main thread has to block forever processing items from the queue.)

On Linux, it isn't 100% clear to me how a CLOCK_REALTIME timerfd reacts to things like the clock being stepped forward over the deadline. We should probably check? But it's not too hard to fix if needed, because you can set a flag on a timerfd asking to be woken any time the clock is stepped, and then do whatever processing logic you want.

I guess the more interesting use case for a clock being stepped is if you have a timer set to "every Saturday at noon" -- in this case if the timer is currently set to the next Saturday – like say it's 2018-01-01, so the timer is set to 2018-01-06 – and then the clock is stepped backwards to 2017-01-01, you don't want to keep sleeping until 2018-01-6, you want to redo the "next Saturday" calculation and reset the timer to a different date (!). Clock change notification is also possible on MacOS by fiddling with Mach ports, as noted here. libdispatch is open source, so you can look at how they do this: src/event/event_kevent.c, search for "calendar_change" (or "CALENDAR_CHANGE"). On windows, you can get this notification from the WM_TIMECHANGE message, but that requires that you be running a window message pump. I'm not sure if there's any other way; the C# SystemEvents.TimeChanged docs make clear that it's based on WM_TIMECHANGE and have special notes on how to get the message pump working if you're in a background service, so maybe there is no better solution.

Brainstorming possible tr- + time-themed names for a trio wall-clock library: treadmill, transience, transient, treacle, trend, trickle

[njsmith]

Here's the twisted bug in case anyone wants to read 11 years of discussion of this topic: https://twistedmatrix.com/trac/ticket/2424

[njsmith]

Though most of the discussion in that twisted ticket seems to be about how to untangle themselves from having decided early on to use a mix of monotonic and wall-clock time. (Not really their fault, but they're using wall clock time for calculations, and then calling select or whatever, and select and friends use monotonic time.) Trio's internal clock is always a monotonic clock, so we don't have that particular issue.

[njsmith]

I poked at this a bit more recently. The most interesting discovery is that the Windows "waitable timer" API appears to handle wall clock waits correctly, without needing to do horrible stuff to get WM_TIMECHANGE messages. In particular, if I set up a waitable timer with a deadline off in the future, then block in WaitForMultipleObjects on the waitable timer, then manually set the clock ahead, then the WaitForMultipleObjects call returns immediately.

This makes this whole project seem much more doable then before: we now have pretty simple/straightforward ways to do this on all three major platforms.

It looks like on macOS, the best way to work with GCD is to use pyobjc: https://pyobjc.readthedocs.io/en/latest/notes/framework-wrappers.html It has wheels for all the versions we care about, and lets us skip trying to figure out how to translate the ObjC docs into ctypes/cffi.

So we could basically just have an await trio.sleep_until_datetime(dt) function, and... that's the public API, done. And internally the system-specific part is a system_wall_clock.raw_sleep_until_datetime(dt), which is guaranteed to only have one call outstanding at a time, might be cancelled, and is allowed to return early. (And there's some generic infrastructure to multiplex multiple sleep_until_datetime calls onto this underlying call.)

Biggest open question for me: integration with mock clocks for testing, and libraries like freezegun.

I think the most generally-useful semantics for mock datetimes are to make them tightly-integrated with the mock/autojump clock:

• By default, they advance in sync (i.e., datetime-clock = monotonic-clock + some offset)
• You can manually change the offset at any point, including backwards

So the way this would look to the user is that manually-advancing or autojump-advancing the monotonic clock will automatically move both clocks in sync, and then there's also a way to manually set just the wallclock time without affecting the monotonic clock. In particular, this means that autojumping will automatically work for arbitrary mixes of monotonic deadlines and wallclock-sleeps!

Looking at the freezegun source, it has to do some super-gnarly stuff to monkeypatch the stdlib to return the fake datetimes, so we probably don't want to reimplement that part ourselves. But, its policies for controlling the fake clock are a bit baroque and don't fit in nicely with our system. So maybe we need to use freezegun, but substituting in our own fake clock policy? There isn't currently a public API for this, but it looks like it would be easy to add: at the end of the day, all of freezegun's machinery already funnels down to a single function call that returns the current time.

The final big question is how to expose the testing machinery: probably we don't want to redefine the existing autojump_clock fixture to start monkeypatching the datetime module! That would be a big change, risky/disruptive, and totally unnecessary for the kinds of cases where people use autojump_clock now. I guess the main cases are (a) run a test with mock trio time, (b) run a test with mock trio time + mock wall time, and in both cases you might want to choose either autojump-or-not, and you need some kind of clock object that you can use to manually control the time? (Or just make them global functions, I suppose.) And what parts of this should live in trio vs pytest-trio vs somewhere else? (I don't think we want to add a dependency from trio->freezegun.)

[njsmith]

Another wrinkle: we do also support FreeBSD now. I just spent a few minutes checking docs and grepping kernel sources, and AFAICT FreeBSD doesn't have any way to get notified when the clock changes. The best you can do is:

• have a thread sit in clock_nanosleep with CLOCK_REALTIME (the kernel does have code to recompute these wakeups internally when the clock changes)
• use a signal to forcibly interrupt that thread when you want to change the deadline

Unfortunately, using signals to wake up threads is very tricky (in particular: as a library, we can't guarantee that any particular signal is available for use; there's also some discussion of this in #174 for interrupting blocking read calls). So probably it would be simpler in the end to just fall back on something basic, like waking up once a minute to check if any realtime timers have expired (and documenting that on these platforms the sleep_until_datetime function only has ~1 minute resolution).

[Edit: could also check whether using EVFILT_TIMER with NOTE_ABSTIME works. It's not very documented, but see here. I'm not seeing any connection between EVFILT_TIMER and the code in settime/tc_setclock that handles waking up clock_nanosleep threads when the clock changes, but I could have missed something, so it wouldn't hurt to verify empirically.]

[joldf]

It's a pity this didn't quite make it to production, especially since it seems like it's mainly because FreeBSD is hard (which is very worthwhile but not the most common case). Perhaps FreeBSD is a rare enough case that it could be dropped, or the problem of picking a signal pushed to the user (you have to make a selection to use the functionality but there's a default and it's only used on BSD)?

Is there any other library that implements something like this just for Windows and Linux, even just a blocking API?

[jakkdl]

It's a pity this didn't quite make it to production, especially since it seems like it's mainly because FreeBSD is hard (which is very worthwhile but not the most common case). Perhaps FreeBSD is a rare enough case that it could be dropped, or the problem of picking a signal pushed to the user (you have to make a selection to use the functionality but there's a default and it's only used on BSD)?

Is there any other library that implements something like this just for Windows and Linux, even just a blocking API?

If you just want a hacky solution, you can write a helper function that uses https://docs.python.org/3/library/datetime.html to calculate the time until a certain date/time and then call trio.sleep.

Looks like there's plenty libraries that has blocking sleeps, e.g. https://pypi.org/project/pause/, https://pypi.org/project/sleep-until/ - though can't vouch for any of them personally.