Pull-based I/O may interfere with fully compositional concurrency

[jdegoes]

These ideas are not fully formed yet, but I wanted to put them out here anyway:

The component model really wants us to be able to treat and think of components as libraries--the cross-language, portable building blocks that make up our applications. However, recently I have found a set of use cases that interfere with this way of viewing components; and, it could be argued, interfere with fully compositional, cross-component concurrency.

The core WASI interface is built on polling, which is pull-based I/O. This seems a logical and sound choice as most operating systems expose pull-based I/O and most applications are built on such interfaces.

However, problems arise when one expects to be able to use one component from another, as one uses one library from another.

To see these issues, first imagine we are building libraries and not components. Let's say, in particular, that we are building a Javascript library to perform auto-complete, which interacts with various indexes and databases. Javascript is both single-threaded, but also concurrent, which makes it a nice fit for wasm-wasi.

In this case, we might have a async function autoComplete(prefix) { } that returns a Promise. This auto-complete algorithm might simultaneously request information from two different APIs using fetch. In addition, the auto-complete algorithm may wish to timeout one of these requests by using setTimeout--if the request has not returned in 60 seconds, then it would be terminated.

This is relatively trivial to do using Javascript, only about 10 - 20 lines of code depending on the approach used. In the end, you have a single-threaded but concurrent function that satisfies its requirements.

There is no issue using such an auto-complete library from another Javascript library. In particular, when you call autoComplete, it returns a promise right away, which you are free to await on right now, if you wish, or you can continue to do other processing and not await on the promise immediately (or ever). In no case does the call to autoComplete semantically "block" your application. Even awaiting the promise does not interfere with other concurrent operations that are happening in your library.

Now, let's instead imagine that we compile our Javascript library to a component, which exposes a auto-complete method. Now we wish to use this component from other components, in the same fashion that we wished to use the auto-complete library from other libraries.

While the component model makes this possible and reasonable to do, in a cross-language way, it turns out that the concurrent nature of the auto-complete component does not compose in the way we would expect with another component.

In particular, if in a second component, we call auto-complete from the auto-complete component, then the event loop of the Javascript component will not finish running until all pending i/o events have been completed. In our case, this means that auto-complete will not return until both fetches return a result, and also the setTimeout callback is invoked.

In other words, unlike when calling a Javascript library, invoking a function on the component instance will block our application for at least a minute. During this minute, we will not be able to invoke any other function on the instance. Why? Because its event loop is already busy handling the protracted execution of the auto-complete function, and will not yield until all pending events have been processed (including the scheduling event created by the setTimeout).

To me, this behavior is quite surprising. It means that even though a component appears to be a library, you can only call one function on an instance at a time, and those invocations have to wrap up fully before you are allowed to call another function. Not only is this unexpected behavior (for me), but it seems hard to reason about, because it is so easy (in Javascript, at least) to "do things later" and "launch concurrent processes" (like fetch) and yet return a value right away.

I could be wrong, but I believe that many developers will expect components to act like libraries, and to expect compositional concurrency, in the sense that the concurrent behavior of two composed components should be the same as the concurrent behavior of two composed libraries.

Now, this appears not to be easy to fix. The fundamental problem appears to come from the pull-based nature of WASI I/O. In order for an event loop to return prematurely (before pending I/O is complete), there would have to exist some exported function in the WASM component, which could receive external events (such as scheduling events or I/O events).

Achieving fully compositional concurrency, as defined here, would seem require something closer to "async I/O" rather than poll-based I/O. This would allow the event loop to yield after it is done processing pending events, which means when no more productive work can occur, and the component instance is waiting on a timer or I/O, then another component may call into it (concurrently), and the execution of such new invocations will be interleaved with the progressing results of prior invocations--which is exactly how concurrent operations behave between two composed libraries (in Javascript or other PL).

This might be a small change to the WASI interface (pull-based to push-based), however, I think it would have quite significant ramifications downstream. Supporting async I/O in some environments (like StarlingMonkey) could be straightforward, but I can see it being much more work in languages whose standard libraries are built on and assume pull-based I/O.

cc @lukewagner