Allow different backends

[vorner]

It would be cool to be able to use something else than mio for the backend. The motivation is, if you have a GUI application with GTK, you would use the GLIB backend and have just one event loop, handling both the UI and the other asynchronous operations.

Currently, if we want to use tokio based networking in a GUI application, we have to run the GUI in one thread and tokio in another, sending data between them. That is a smaller problem than in other languages, but it would still be nice to be able to use just one.

Other languages do something like this (eg. perl's AnyEvent), so it might be possible. On the other hand, it might be harder in Rust, as the other languages are usually dynamically typed.

One possible direction could be for the Handle to be a trait instead of a concrete type and all the nice abstractions built on top of tokio could use that.

[alexcrichton]

This is sort of the job of futures rather than tokio core, where futures provides a backend-agnostic method of working with future-related data. Basically all of tokio-core is architected to work with mio, and swapping in a backend would amount to an entire rewrite anyway. If you're always working at the futures layer though you'll continue to be abstract over all event loops in theory.

[vorner]

I suspect a little misunderstanding here.

First, this abstraction is obviously not handled by futures. Let's say we have a library that can be a http client (I know Hyper is not there, tokio-wise, yet, but let's pretend it is). Because it needs to create a TcpStream, it needs tokio-core. Which basically kills the whole compatibility with another event loop.

And I wasn't exactly suggesting to directly support other backends. My idea was more in the sense to separate the interface and implementation, so someone else could write another event loop with compatible interface.

A concrete idea would be:

• Split the event-loop agnostic parts into a separate crate (eg. the Io trait, Codec, etc).
• Provide some kind of trait, like:

trait ReactorCtl {
  type TcpStream,
  type TcpListener,
  ...
  fn create_tcp_stream(addr: ...) -> Future<Self::TcpStream, io::Error>,
  ...
  fn spawn(...),
}

This could be implemented both by reactor::Core and reactor::Handle. Then, the user could pass a handle to Hyper or other library to do its thing. And the library would still work if someone provided another implementation of that trait, backed up by glib or whatever else.

Obviously, this idea is very much in flux, but I think it could be possible to design it in a way Core doesn't have to change and others would have the option to provide different implementation.

[aturon]

@vorner I think a good place to start here would be to prototype such an alternative back-end directly with futures. It's usually better to have a few different concrete instances in mind before trying to abstract out to a trait.

[carllerche]

@vorner I would say that Hyper doesn't actually need to take a TcpStream. It would be sufficient for it to take T: Read + Write, which would abstract away Tokio for the most part.

tokio-proto currently is fairly bound to tokio-core, but there probably isn't a strict need for this to be true.

[vorner]

@aturon: I'm afraid I don't have the time and skill in Rust to do that just now (I think the hairiness of my attempt to write an alternative event loop on top of mio could attest to that). But my point wasn't to propose a concrete abstraction, just to clarify with an example the general direction of my thinking, that the event loop could implement some trait (not necessarily this concrete one) which other event loop could also implement sometime in the future. I was kind of hoping someone with more experience with designing Rust APIs could come up with something better, if this issue got considered.

@carllerche I must admit, I didn't have a look at Hyper's interface (I tried to look and found only the server-side in tokio-hyper). I was thinking about something like this:

let future_of_response = http_download(handle, "http://example.com");

Such function doesn't directly take TcpStream or T: Read + Write, but it creates that internally (possibly more than one, in case it does retries or follows redirects). The current interface leaves no way to pass it anything but tokio_core::reactor::Handle, because this is a concrete type, which leaves no room for placing it on top of another backend.

Indeed, if I really wanted to write the http_download in a backend-independent way, I could pass a closure that could create the corresponding type for TCP socket when called, but that would be awkward to use:

let future_of_response = http_download(|address| {
  TcpStream::connect(address, handle)
}, "http://example.com")

So, my motivation to open this issue was to start a discussion about if it is worth considering as a goal, to be possible to replace the Core with something else, or if it is deemed uninteresting. It's not that I knew how to reach that goal, unfortunately ‒ I'd like to help where I can, but I'm afraid I can do only a little thing here or there until I get more comfortable with Rust.

[antoyo]

@vomer Relm is my attempt at combining GTK+ and futures/tokio.

You can also look at the http example to see how an asynchronous HTTP request is done within a GTK+ application written with relm.

[vorner]

I did have a look. It definitely can solve some problems, but looking inside, you seem to be doing some kind of busy loop switching between tokio and gtk+. Not that I'd have a better solution now, but this doesn't seem very clean.

[antoyo]

Indeed, it is not clean. I talked on the #rust GNOME channel and one solution that came up was to put the tokio loop in another thread and communicate via channels. But I'm not sure about how to integrate this channel communication with GTK+.

@alexcrichton Do you have any example of integrating the tokio event loop with another event loop? The issue with using only Future and not tokio is that some tokio-based library needs a Handle.

[alexcrichton]

@antoyo unfortunately no I don't have many existing examples. I'd recommend one of two strategies though:

• If you must run a tokio-core event loop, I'd run it in a separate thread and communicate with channels. You'll need to implement a mechanism to send messages to the GTK event loop though.
• Ideally though tokio-core would not be needed for an event loop (as you've already got an event loop). In that case I'd recommend pushing libraries to primarily work with the futures crate (e.g. Stream and Sink traits).

In general yeah libs currently need Handle from time to time, but that's typically basically just for spawn (if they don't need concrete I/O objects) and we've been investigating making this generic as well to interoperate with foreign event loops better.

[antoyo]

Thanks.

@vomer I updated relm to use another thread instead of interleaving both event loops as suggested by some people (including @alexcrichton). If anybody has any suggestion to improve relm (especially its interaction with futures/tokio), feel free to open an issue for this crate. Hopefully, relm could serve as a real-world example of integrating an event loop with tokio.

I am looking forward for better integration of another event loop with tokio.

[alexcrichton]

Nice @antoyo!

Note that with the recent addition of the tokio-io crate most librarys may not end up requiring tokio-core. For example the tokio-tls crate only requires tokio-core as a backcompat support, not for actually servicing connections. Similarly the tokio-proto crate really only needs the ability to spawn a task (https://github.com/alexcrichton/futures-rs/issues/313) and otherwise doesn't require tokio-core at all.

In that sense I think that if your event loop can provide concrete types that implement the traits in tokio-io then we've got a clear path forward to eliminating dual event loops by placing everything onto the GTK event loop for you. In other words, the ecosystem should largely be capable of being generic with tokio-io which doesn't imply a runtime, but the one missing piece right now is the Spawn trait (in one form or another)