External runtime integration

[bugaevc]

I would like to be able to use smol in software that already uses other runtimes. By that, I don't mean tokio or async-std; I mean completely different runtimes (usually called "I/O loops", "event loops" or "run loops") usually provided by an OS or by a toolkit. A few examples of such runtimes:

• libdispatch
• CFRunLoop
• glib main loop
• Wayland event loop

I'm not suggesting smol should add explicit support for each and every external runtime; I'm thinking of a more generic scheme, akin to what the Wayland event loop does (see wl_event_loop_get_fd() / wl_event_loop_dispatch() / wl_event_loop_dispatch_idle()).

The general idea is I would want the external runtime, rather than smol's reactor, to do the waiting on I/O part. Then, once some I/O can be performed on one of the file descriptors registered with smol, the external runtime would call back into smol, letting it handle the event.

This basically means smol should provide the API to get its epoll/kqueue fd (to be registered with the external event loop), and to "dispatch" events when the fd becomes readable (so, run its reactor & executor until it has nothing to do, then return instead of blocking).

[sdroege]

The usual approach for this would be to run things in different threads and then integrate them with each other via channels.

* glib main loop

Note that this implements (in the glib crate) a full futures runtime too. I don't think integration between smol (the runtime/reactor parts around epoll and friends) and glib would make sense, but things like the Async type could get an implementation that works with the glib runtime and that wouldn't even be very hard and IMHO a good idea for cross-runtime interoperability. I have that somewhere on my todo list, to experiment with that :)

I assume the situation is more or less the same for CFRunLoop, it's very similar to the GLib main context / loop.

* Wayland event loop

There are ways to integrate Wayland with other event loops via an fd (see e.g. GTK/Wayland on top of the GLib main loop), so it should be possible to integrate into anything that allows polling fds in one way or another... so also smol with the epoll-based reactor.

[bugaevc]

The usual approach for this would be to run things in different threads and then integrate them with each other via channels.

That's an unfortunate workaround you'd have to resort to when the runtime (smol) doesn't support proper external runtime integration. I can't always afford to spawn threads, especially if I'm a library or a plugin used in a larger program that already uses another runtime. As I said I'd like smol to support wl_event_loop-style integration, where you seamlessly run one runtime inside another by registering inner runtime's epoll/kqueue fd with the outer runtime.

Note that this implements (in the glib crate) a full futures runtime too.

Yes, but the reality is a lot of software already uses a different runtime. In particular I'm thinking of libraries that use async-std, which now uses smol internally. I would want those libraries to integrate into the glib runtime, which is why I'd want to get my hands on smol's epoll/kqueue fd and register it with glib main loop.

There are ways to integrate Wayland with other event loops via an fd (see e.g. GTK/Wayland on top of the GLib main loop), so it should be possible to integrate into anything that allows polling fds in one way or another... so also smol with the epoll-based reactor.

Yes, that is already possible, exactly because wl_event_loop supports being integrated into other runtimes. I propose that smol starts supporting that in a similar way too, so that it becomes possible to, for instance, integrate smol into the Wayland (or any other) event loop, rather than the other way around (which is again already possible in case of Wayland event loop).

[sdroege]

AFAIU that's the goal (but @stjepang would know for sure) in the end.

To some degree you could already achieve that by making it possible to implement something to allow using the Async type on arbitrary runtimes instead of going to the epoll-based reactor in smol, having library crates only use that (plus the generic interfaces in futures and std) and none of the other API exposed by smol (e.g. spawning a task, there are discussions about a generic API for that elsewhere).

It's one of the many building blocks needed for building runtime-agnostic futures APIs.

I don't think it would make sense for smol to provide the integration with other runtimes though: that's something these other runtimes would have to provide in the end (for making things like the Async type work on their own reactor instead of smol's current reactor). I think for that to work some more abstraction is needed in smol though, and probably splitting out things into more crates and making smol make use of these then.

Addition: Also I believe in the future no library crates should depend on async-std or tokio or smol for that matter (unless they are actually specific to them), but instead use common abstractions that allow usage with any runtime. So being able to poke at the epoll reactor inside smol to integrate the fds in e.g. the glib main context seems a bit backwards to me.

[sdroege]

Btw, just to be clear, I completely agree with your overall goal here. I'm one of the maintainers of the glib crate and a GLib developer, and I also have the need to use custom async runtimes in various applications of mine (where neither async-std, tokio, glib nor smol's whole runtimes would fit well).

I think at this point the overall problem is clear to the Rust community (and if it's just because mixing async-std and tokio is annoying), and smol's design (especially the Async type, but also the overall runtime/reactor design that makes it a great fit for basing custom runtimes on it) is a good step forward here but there's still a lot more to do.

[bugaevc]

It's one of the many building blocks needed for building runtime-agnostic futures APIs.

Also I believe in the future no library crates should depend on async-std or tokio or smol for that matter (unless they are actually specific to them), but instead use common abstractions that allow usage with any runtime.

I'm all for runtime-agnostic futures APIs, but again, lots of code uses async-std today and is not becoming runtime-agnostic any time soon.

So being able to poke at the epoll reactor inside smol to integrate the fds in e.g. the glib main context seems a bit backwards to me.

Let me describe a use case I have in mind more concretely. Say I'm writing a plugin for an existing application that already uses a runtime (be it glib, libdispatch or something else that can watch over fds). I want my plugin to use some other library that already hard-depends on smol (perhaps transitively through async-std). That other library knows nothing about my plugin, the external runtime or anything, it just blissfully uses async_std::net::TcpStream as if it was the only thing running in this OS process/thread (narrator voice: it isn't). I, the plugin author, know about both the library and the external environment, and I could perform some setup (namely, add smol's reactor fd to the external runtime) to make that library automatically work for me.

I don't think it would make sense for smol to provide the integration with other runtimes though: that's something these other runtimes would have to provide in the end (for making things like the Async type work on their own reactor instead of smol's current reactor).

It's also unrealistic to expect other runtimes to support smol (or know about its existence). We have to aim for a generic interface that works for stacking any runtimes on top of each other. The interface is:

• a runtime supports watching over a given fd (this can be used to integrate other runtimes into it, among other uses)
• and it also supports exporting its own fd and dispatching events when it's told the fd is ready for reading (this can be used to integrate it into other runtimes).

The Wayland event loop is a perfect example, as it implements both. async-std currently implements none. smol only implements the first one (Async::new()), but not the second one, which is what I'm asking for :smile:

implement something to allow using the Async type on arbitrary runtimes instead of going to the epoll-based reactor in smol,

That — having smol::Async register the fd with any reactor of your choice (instead of always using smol's own reactor) — would also be pretty neat. But it'd be enough to have it still use smol's own epoll/kqueue, but let me add that to another reactor.

[polachok]

and it also supports exporting its own fd

Exporting fd seems tricky to-do in cross-platform way. It would work with proper epoll, but what about wepoll?

and dispatching events when it's told the fd is ready for reading (this can be used to integrate it into other runtimes).

So, something like turn in tokio-core?

[seirdb]

I’m in need of something similar; a framework with it’s own runtime, but which should also be easily integrated in an other (i.e. libevent, glib, ...) framework.

In the end, it’s not sufficient to export a single fd which the other runtime can monitor (in either runtime enviroment), as you’d need to share all fd’s that the event loop needs to monitor. Even something as simple as monitoring a single UDP connection and handling a timer (through a timerfd on unix f.e.) already requires 2 fd’s to be exposed to the external event loop.

[bugaevc]

Exporting fd seems tricky to-do in cross-platform way. It would work with proper epoll, but what about wepoll?

I have very little idea about Windows internals, sadly :slightly_frowning_face: If there's nothing similar to this in the Windows world, this API can always be made Unix-specific, much like the rest of fd APIs already are.

So, something like turn in tokio-core?

Yes, like that. Or like the existing smol::run() that returns at this point instead of blocking:

https://github.com/stjepang/smol/blob/6dabee5326bab37b9560fb62e24c46816b8fd083/src/run.rs#L173

[bugaevc]

In the end, it’s not sufficient to export a single fd which the other runtime can monitor (in either runtime enviroment), as you’d need to share all fd’s that the event loop needs to monitor. Even something as simple as monitoring a single UDP connection and handling a timer (through a timerfd on unix f.e.) already requires 2 fd’s to be exposed to the external event loop.

With APIs such as epoll & kqueue (as opposed to poll & select), it's possible to nest polling: you can actually watch one top-level fd (the epoll fd / kqueue fd), and it becomes readable when it gets any of the events (for many other fds) you've registered with it.

To quote epoll man page:

Is the epoll file descriptor itself poll/epoll/selectable? Yes. If an epoll file descriptor has events waiting, then it will indicate as being readable.

[seirdb]

With APIs such as epoll & kqueue (as opposed to poll & select), it's possible to nest polling: you can actually watch one top-level fd (the epoll fd / kqueue fd), and it becomes readable when it gets any of the events (for many other fds) you've registered with it.

Very interesting, I wasn’t aware of this.

This means that it should be sufficient to expose the epoll fd, and have a single-iteration function similar to the one from tokio mentioned above.

[bugaevc]

...which is what I've been asking for in the first place 😀

[ghost]

@bugaevc I'm working on exposing the reactor internals for integration with other event loops.

[ghost]

Ok I created these crates:

• https://docs.rs/multitask
• https://docs.rs/async-io

With multitask, you can create an Executor. Each worker thread then creates a Ticker for running tasks and calls .tick() in a loop. This allows you to manually drive the executor step-by-step inside other event loops.

With async-io, you can create a Parker for waiting on I/O events (optionally with a timeout). Note that there is also a fallback background thread driving the reactor in case you choose not to drive the reactor. But if you do, anytime you call park(), the current thread will wait on I/O events and wake wakers as soon as an event occurs, unless another thread is already parked and waiting on I/O events.

@bugaevc Do those two crates resolve this issue?

[bugaevc]

Sounds pretty cool — though it looks like the docs could use some improvements & more explanations. For example, park() docs say nothing about waiting for I/O, or about there being a single global reactor.

So if I understand correctly, tick() means running the tasks that are ready to make progress, and park() means waiting for I/O, or for an explicit unpark(), and both do that once (not in a loop). That checks off my "run its reactor & executor until it has nothing to do, then return instead of blocking" requirement.

What's still missing (or at least I haven't found it) is a way for me to get reactors epoll/kqueue fd, to be able to pass it to another reactor/runtime.

[ghost]

Your understanding of tick is correct. Docs do need some improvement :)

I wonder, is taking the epoll or kqueue fd actually useful? Note that the reactor keeps a mapping between event userdata (which is an u64 containing Source ID) and wakers interested in events. So if we share the epoll or kqueue fd between multiple reactors, we must also share this mapping.

[bugaevc]

I wonder, is taking the epoll or kqueue fd actually useful? Note that the reactor keeps a mapping between event userdata (which is an u64 containing Source ID) and wakers interested in events. So if we share the epoll or kqueue fd between multiple reactors, we must also share this mapping.

Oh, that's not what I mean by using the epoll/kqueue fd. I don't mean to register more fds with it (such as with kevent64(kqueue_fd, ...)) on my own, bypassing smol's/async-io's reactor. I mean adding that epoll/kqueue fd itself to another reactor, just to be able to wait for it becoming readable. Once the epoll/kqueue fd becomes readable — that is, once there are events queued on some of the sources registered with this epoll/kqueue — we can call park() and be sure it wouldn't actually block.

(That is, the underlying epoll_wait() and kevent64() wouldn't block — park() shouldn't block in that case either, right?)

[ghost]

I see, thanks for clearing that up! If I may push a bit further: why wait until the fd becomes ready and then park? Is it not better to wait for Executor/LocalExecutor to invoke your notify callback? That’s how we know a task has been scheduled in the executor and we may proceed by “ticking” one step forward.

[bugaevc]

Is it not better to wait for Executor/LocalExecutor to invoke your notify callback?

I haven't actually noticed those notify callbacks when skimming through multitask API! And looking at it now, Executor::ticker() docs don't explain what that callback is supposed to be or when it's invoked. And the implementation of tickers is apparently also more complex than it seemed from the surface API — a ticker can "sleep" (block?), and be notified by other tickers...

Perhaps you could clarify how this all works and is supposed to fit together?..

[ghost]

I published a new version of this crate, which should make this a bit simpler: https://docs.rs/async-executor/0.2.0/async_executor/

It provides an Executor that you can drive manually using run(). If you want to perform a single "tick" and run just a couple of tasks, you can do executor.run(future::yield_now()). The API is really minimal and it's all conceptually simple.

Let me know if this is something that might work for you!

[bugaevc]

If you want to perform a single "tick" and run just a couple of tasks, you can do executor.run(future::yield_now()).

That's clever — but does it actually guarantee that any tasks that can make progress will be run until they all make all the progress they can without blocking?

Let me briefly describe the use case once again.

There's some external event loop; I fetch the epoll/kqueue fd that the smol reactor is using (still need an API for that!) and register interest for readability of that fd with the external loop. When the external loop calls me back, saying that fd has become readable (meaning — since it's an epoll/kqueue fd — that there are now events queued on it), I want to tell:

• To the smol reactor, that it should now fetch the queued events (such as with epoll_wait()), because now that we know that there are, in fact, some queued events, we are sure the epoll_wait() call won't block. Fetching the events will naturally run callbacks associated with those events, and so wake some tasks.
• To the smol executor, to now run all the tasks that can make progress (primarily those that were just woken up by the reactor) until they all cannot make any more progress (without blocking).

At this point, I return from my callback back to the external event loop, and wait for it to tell me about my epoll/kqueue fd becoming readable again.

[ghost]

I added two more methods to Executor to make this easier: https://docs.rs/async-executor/0.2.1/async_executor/struct.Executor.html

• fn try_tick(&self) -> bool
• async fn tick(&self)

You can keep calling try_tick() in a loop to process all scheduled tasks. When there are no more tasks, you can use tick() - and that future will be woken when the next task is scheduled.

[notgull]

I think that async-executor is good enough now that it can be pretty easily integrated into other systems. async-io isn't there yet; see smol-rs/async-io#124.

I don't think this is possible on Windows. Our notification mechanism on Windows is an I/O Completion Port, and you can't really register an IOCP into another IOCP.

[bugaevc]

I still have very little idea about Windows internals, but recently this post has made rounds, suggesting that on Windows some "Auxillary Function Driver" is used to actually poll all the sockets for readiness, and then that AFD is what's used with IOCP. In that case, wouldn't it make sense to export this AFD handle to enable adding it to the outer event loop?

[notgull]

In that case, wouldn't it make sense to export this AFD handle to enable adding it to the outer event loop?

Unfortunately, we can't do that. The way smol works, and the way other platforms like libuv work, the polling implementation require a not-insignificant amount of code on top of IOCP/AFD. This code handles completions, which are exclusively tied to the associated IOCP. This makes it not compatible with other code from, say, libuv. In addition, other runtimes like GLib don't use IOCP/AFD at all. If I recall correctly, GLib uses WSAPoll under the hood, which neither AFD nor IOCP would be compatible with.

In addition, not all events are carried over AFD. In polling, notifications are posted straight to the IOCP without going through AFD at all.

While you can put an I/O Completion Port into WaitForMultibleObjects, it's not portable between Windows versions, and most sane runtimes nowadays don't use it. Unfortunately the "blessed" way of integrating event loops together on Windows is to just run them in different threads. I'm pretty sure this is what Electron does.

[notgull]

https://github.com/smol-rs/async-io/pull/125 is now merged, so I believe that this issue is taken care of.