Switch AsyncFD to use kEVENT on OSX / *BSD's for Asynchronous thread notification?

[elcritch]

Great work getting https://github.com/status-im/nim-chronos/pull/406 in! It'll be handy.

It looks like Chronos is using AsyncFD which reminded me of a PR I'd done a while back on MacOSX for in the Nim stdlib io_selectors to use kEVENTs for async thread triggers. It might be useful for Chronos thread async as well on *BSDs.

On *BSD platforms AsyncFD looks to use socketpair as well. On many systems that's normally similar to a pipe that the Nim stdlib uses for asyncfds on *BDSs. pipe works reliably, but when you push lots of events through the timing can become a quite erratic. socketpair probably also goes through caching & network buffers, which like pipe, might behave oddly under load.

Linux's eventfd implements an actual semaphore object which performs much better. On *BSD's the kqueue mechanism has user events that are similar to eventfd. Complete aside: Zephyr RTOS also provides eventfd.

In my case, I was using it for async thread notifications in Fidgetty and was a bit annoying. Sometimes it would take +500ms to notify when under heavy load (as in thousands of events being triggered per second). It wasn't an issue on Linux, or under lighter loads. Granted, I didn't do quantitative timing tests but the effect it pretty visible in a 120hz GUI.

If there's interest I could possibly look into it. A lot of dev's use MacOS for testing / development so it may be useful in order to make dev performance match Linux / Windows.

Background

It appears on MacOS that pipe (which Nim AsyncFD uses) and socketpair (which chronos uses) may be implemented distinctly according to this LWN article. The socketpair may not exhibit the same lag / jitter issues.