Is it easy to add further examples on how to use async await together with this example?

[liufuyang]

Feels like we are close to use rust standard future and even using the async await syntax. Do you plan to do some further work on this for your example? :)

[cfsamson]

I'm actually working on explaining Futures and async/await right now (if I can manage it will be in 200 lines of Rust as well, but they might end up in two small books). I started out with the same idea as you have, but thinking more closely about it it's not a very good example to use:

We could:

1. Let the user write code in a thread as it was synchronous code. Once a blocking call is made we instead yield to the scheduler which parks our green thread

2. The scheduler then creates a Waker which is passed to the Reactor which will respond to any incoming I/O event from the OS.

3. When the reactor calls Waker::wake() in response to some I/O Ready event, it signals to our scheduler that our thread can continue running the thread.

This is essentially what Go does and most other Fiber implementations do. However, since Futures and async/await aims to provide an ergonomic way for users to write code it's not really applicable here since the user can write normal synchronous code in our green threads, it's no point for them to use async/await on top of that.

We could use async/await in our implementation though, but then we'll actually need "two" runtimes. One to drive our futures and one to drive the green threads. All we need is some sort of Waker really. At least this is my current thoughts about this.

I'll see if I can do something fun with this, but right now I'm explaining Futures in a much simpler way. At least I hope to.

[liufuyang]

Sounds great. But I didn't understand the part of "it's no point for them(users) to use async/await on top of that."

I thought it would be a good idea eventually to evolve this example to a super simple runtime very much like async-std or tokio (but without the actual io support)? So it would be helpful for readers to learn how to implement a runtime on their own, or it would help reader understand how async-std/tokio kind of stuff is made together with the rust async away feature?

It seems that out there there is not a single example you can run yet with rust async syntax without using some crates such as futures/async-std/tokio. I thought it would be cool that we extends this example to allow users spawn tasks with async syntax.

Then this example would look sometime completed. Or am I misunderstood something :P

[cfsamson]

It's possible to use this as a runtime, and it would be a cool one, but we need to choose whether the user would use Futures and asait/async, or just write normal synchronous code in our green threads.

Let's take the example of reading from a socket which is a blocking operation

thread::spawn(|| {
    let mut text = String::new();
    socket.read_to_string(&mut text);
    t_yield();
    // at this point, instead of blocking we suspend the thread in our runtime
    // behind the scenes (invisible for the user), our scheduler (or `Executor`) calls a Reactor 
    // and register interest in a `Read` event on this socket. Once ready, the `Reactor` notifies  the
    // scheduler that data is ready for us to read so it switches back to resume our thread.
    println!("{}", text);
}

In this example, we won't use futures inside our thread...

Now compare that to Futures and async/await in Rust:

fn main() {
    let future = async {
        let mut text = String::new();
        socket.read_to_string(&mut text).await;
        // Rust will "recompile" this into a state machine in which this _suspended-awaiting-read_ 
        // will be one of the states the future can be in.
        println!("{}", text);
    }
    let mut rt = tokio::runtime::Runtime::new().unwrap();
    rt.block_on(future);
}

As you see, Rusts Futures (and thereby async/await) solves the same problem our threads solve from the users' perspective. An ergonomic way to write asynchronous code.

We could in theory make the green threads an Executor and spawn our futures to the green thread runtime where they get polled, and then we yield if the result is Pending, but that wouldn't take advantage of the threads fully since they will function as a very fancy loop, a simple loop would do the same.

On the other hand, my first draft of a minimal example of a Reactor, Executor and runtime is less than 200 lines of code alone. You can check it out here: https://github.com/cfsamson/examples-futures/blob/ProperWakerClean/src/main.rs

Note that we crate our own futures in that example. The next step is to use std::futures::Future which can be run on any runtime (not only our own). You can see the changes here: https://github.com/cfsamson/examples-futures/blob/TokioFut/src/main.rs

(just remember that we use tokio here so we can remove our own executor).

FYI: this is just a first draft and still work in progress.

[liufuyang]

That is pretty cool, will check them out 👍 And thanks for the detailed reply. I think I get what the later part means. Just the part where you saying "we won't use futures inside our thread" with example like:

thread::spawn(|| {                               
    let mut text = String::new();
    socket.read_to_string(&mut text);   // line 3
    t_yield();                          // line 4
    println!("{}", text);
}

wouldn't line 3 will already block the running thread reaching line 4? So this wouldn't really work as some async call? Perhaps you meant that read_to_string() would be some function (either really do io or fake method) the user implements, which also havs a t_yield(); somewhere in it's block?

Sorry I am still a bit confused with all this. Perhaps I should wait when you have some other gitbooks read on those topics to learn :)

[cfsamson]

No problem. It's not that easy either :) And bear in mind that I'm thinking out loud here, so there might be some places I'm not 100% correct.

I meant that socket.read_to_string(&mut text); is something we implement so it's not blocking (i.e. we don't use the Socket in stdlib, but create our own and provide our own Read implementation for that).

Our implementation could instead register interest in an event on this socket with our Reactor and pass a Waker to wake up the thread again. That way it wouldn't block our OS-thread.

Most likely we would call t_yield inside that function as well so the user will not even "know" that their thread yields control to the scheduler, they just write code as usual. This is the great advantage of using green threads.

I have actually two more books. They're not that popular and are way too heavy (read they're a bit boring to read maybe) for most, but they do go in depth in a lot of these topics: https://cfsamson.github.io/book-exploring-async-basics/ https://app.gitbook.com/@cfsamsonbooks/s/exploring-epoll-kqueue-and-iocp/

I do however hope that the next one, about futures and executors will be a lot shorter and more fun to read.