Enabling concurrency in for await loops

[jamiebuilds]

Some code from the recent Deno 1.9 release blog post:

for await (const conn of Deno.listen({ port: 4500 })) {
  (async () => {
    for await (const { respondWith } of Deno.serveHttp(conn)) {
      respondWith(new Response(body));
    }
  })();
}

This code is making use of a self-invoking async function in order to run some async code concurrently.

Using async do you could replace this with:

for await (const conn of Deno.listen({ port: 4500 })) {
  async do {
    // ...
  }
}

However, this still has a problem. If an error is thrown inside of the async do {} statement, it's not being handled anywhere because the promise returned by async do is never being await-ed. I think this would be a surprising behavior for users. Setting up an http server with Deno could be the first time someone uses JavaScript, and this could be among the first code they write.

Attempting to add error handling to the above code is also non-trivial:

await new Promise(async (resolve, reject) => {
  for await (const conn of Deno.listen({ port: 4500 })) {
    (async do {
      // ...
    }).catch(reject)
  }
  resolve()
})

Concurrency primitives could be pushed into IterTools-like APIs such as:

await AsyncIteratorTools.mapConcurrently(Deno.listen({ port: 4500 }), async conn => {
  // ...
})

However, with syntaxes like for await and async do being developed, I think it makes sense to explore this within syntax space

For the purpose of having a clear discussion in comparison to async do {}, I'll create a new syntax spawn {} (I have no attachment to the keyword/syntax)

for await (const conn of Deno.listen({ port: 4500 })) {
  spawn {
    for await (const { respondWith } of Deno.serveHttp(conn)) {
      respondWith(new Response(body));
    }
  }
}

Differences:

• spawn looks for the nearest async context (either an async function or for await or the module)
• spawn creates a promise that is implicitly await-ed in a non-blocking fashion, but if it rejects, an error is thrown immediately
• spawn would only be valid in places where await is valid

This would mean that you can spawn async tasks inside of any async function or for await loop and trust two things:

• All of your spawn'd tasks will be resolved before your code continues (after you've called an async function, or you've looped over an async iterator with for await)
• Errors won't go unhandled

let sleep = ms => new Promise(resolve => setTimeout(resolve, ms))

let items = []

async function runTasks() {
  spawn { await sleep(200); items.push(1) }
  spawn { await sleep(300); items.push(2) }
  spawn { await sleep(100); items.push(3) }
}

await runTasks()
console.log(items) // [3, 1, 2]

spawn would continue to return its promise so that you could choose to explicitly await it whenever you want.

let promise1 = spawn {}
let promise2 = spawn {}
await promise1
await promise2

Thoughts?

[bakkot]

It's definitely a space which could use some additional tools.

spawn creates a promise that is implicitly await-ed in a non-blocking fashion, but if it rejects, an error is thrown immediately

I think of await as inherently blocking. What does it mean to await in a non-blocking fashion?

[jamiebuilds]

I think of await as inherently blocking. What does it mean to await in a non-blocking fashion?

I wasn't quite sure how to explain that. I meant the abstract Await() operation, but maybe PerformPromiseThen() is more correct to say.

In short, the promise created by spawn {} would always have .then(onFulfilled, onRejected) called on it immediately. onRejected would cause an immediate error to be thrown, but the code after spawn {} would not wait for onFulfilled to be called. Instead the parent async function or for await would wait for onFulfilled (or onRejected) to be called.

[bakkot]

Instead the parent async function or for await would wait for onFulfilled (or onRejected) to be called.

When would it do the waiting?

To be concrete, here's one possible sugar for what I think you're suggesting. Is this what you mean?

for await (const conn of Deno.listen({ port: 4500 })) {
  spawn {
    for await (const { respondWith } of Deno.serveHttp(conn)) {
      respondWith(new Response(body));
    }
  }
}
doStuff();

->

let deferred = [];
for await (const conn of Deno.listen({ port: 4500 })) {
  deferred.push(async do {
    for await (const { respondWith } of Deno.serveHttp(conn)) {
      respondWith(new Response(body));
    }
  });
}
await Promise.all(deferred);
doStuff();

(Edit: though, I guess that doesn't do the error handling quite like you suggest...)

[jamiebuilds]

I think this is a bit closer to get the error handling right:

let rejectFn

await Promise.all([
  new Promise((_, reject) => {
    rejectFn = reject
  }),
  async function () {
    let deferred = []

    for await (const conn of Deno.listen({ port: 4500 })) {
      deferred.push((async do {
        for await (const { respondWith } of Deno.serveHttp(conn)) {
          respondWith(new Response(body));
        }
      }).catch(rejectFn))
    }

    await Promise.all(deferred)
  }(),
])
doStuff();

[bakkot]

So it wouldn't actually stop the loop if one of the steps threw, but the promise for the containing function would reject? I'm not sure that's all that intuitive. It's an interesting thought, though.

What about for uses of spawn outside of for await? When would the waiting happen?

[jamiebuilds]

So it wouldn't actually stop the loop if one of the steps threw, but the promise for the containing function would reject

Oh, I hadn't considered that in my de-sugar-ing... Here's an updated example. This is always going to be a race, but based on the conversations of promise cancellation racing, I think that's to be expected.

let rejectFn
let rejected = false

await Promise.all([
  new Promise((_, reject) => {
    rejectFn = reject
  }),
  async function () {
    let deferred = []

    for await (const conn of Deno.listen({ port: 4500 })) {
      deferred.push((async do {
        for await (const { respondWith } of Deno.serveHttp(conn)) {
          if (rejected) break
          respondWith(new Response(body));
        }
      }).catch(error => {
        rejected = true
        rejectFn(error)
      }))
    }

    await Promise.all(deferred)
  }(),
])
doStuff();

What about for uses of spawn outside of for await? When would the waiting happen?

Before:

async function fn() {
  let step = 0

  spawn {
    step = 1
    await doSomethingAsync()
    step = 2
  }

  return step
}

After:

async function fn() {
  let deferred = []
  let returned
  let step = 0

  deferred.push(async do {
    step = 1
    await doSomethingAsync()
    step = 2
  })

  return (returned = step, await Promise.all(deferred), returned)
}

The result of await fn() would be 1, the function fully evaluates including its return before waiting for any remaining promises.

[jvcmarcenes]

I think that the point of async do { } is to enable asynchronous code (use of the await keyword) inside of do { } blocks If the do { } block is already inside an asynchronous context, why not just await do { }. This would wait for the evaluated promise in the do block to resolve

And for error handling, how about:

async do {
  await sleep(2000)
  throw "error"
} catch (err) {
  console.log(err)
}

Now, if the point of spawn { } is to run asynchronous code in a non-blocking fashion, I think it goes far beyond this proposal and should be made it's own proposal (which I am in favor of)

[ljharb]

@jvcmarcenes because that wouldn’t let await inside the do block await the outer async function.

[jvcmarcenes]

@jvcmarcenes because that wouldn’t let await inside the do block await the outer async function.

I'm not sure I follow? what do you mean by await inside the do block await the outer? Why wouldn't the await do { } wait for the do promise to resolve?

[bakkot]

I think that the point of async do { } is to enable asynchronous code (use of the await keyword) inside of do { } blocks If the do { } block is already inside an asynchronous context, why not just await do { }.

It's better to think of async do as introducing a new asynchronous context. Even inside of an asynchronous function, if you want to do two things at the same time, you have to resort to awkwardness like

let results = await Promise.all([
  (async () => {
    let result = await fetch('one');
    return await result.json();
  })(),
  (async () => {
    let result = await fetch('two');
    return await result.json();
  })(),
]);

With async do, that could be

let results = await Promise.all([
  async do {
    let result = await fetch('one');
    await result.json()
  },
  async do {
    let result = await fetch('two');
    await result.json()
  },
]);

The point of async do is to make it simpler to await without blocking the outer function. That's useful in synchronous contexts (because you can't await at all), but also in asynchronous ones, as above. It's actually a very different thing than await do {}.

[jvcmarcenes]

The point of async do is to make it simpler to await without blocking the outer function. That's useful in synchronous contexts (because you can't await at all), but also in asynchronous ones, as above. It's actually a very different thing than await do {}. I see, but I never said that async do { } and await do { } where the same

the point of await do would be to await for the do block to finish asynchronous execution. And since the the asynchronous do is awaited, it also provides syntax for handling errors.

the await do would have no intetion to solve the following example:

let results = await Promise.all([
  async do {
    let result = await fetch('one');
    await result.json()
  },
  async do {
    let result = await fetch('two');
    await result.json()
  },
]);

Perhaps I am just confused with the proposal of spawn { } in the first place, and if that is the case, I apologize for wasting the time

However, this still has a problem. If an error is thrown inside of the async do {} statement, it's not being handled anywhere because the promise returned by async do is never being await-ed. I think this would be a surprising behavior for users. Setting up an http server with Deno could be the first time someone uses JavaScript, and this could be among the first code they write.

Attempting to add error handling to the above code is also non-trivial:

await new Promise(async (resolve, reject) => {
  for await (const conn of Deno.listen({ port: 4500 })) {
    (async do {
      // ...
    }).catch(reject)
  }
  resolve()
})

I think that await do could solve for this problem, with the following syntax:

for await (const conn of Deno.listen({ post: 4500 })) {
  await do {
    // ...
  } catch (err) {
    // ...
  }
}

[bakkot]

A regular do does not create a new asynchronous context. When you await inside of it (or, of course, if you await its result), you're blocking the outer function. The point of async do (and of this thread's spawn) is that you can await without blocking the outer function.

[jvcmarcenes]

A regular do does not create a new asynchronous context. When you await inside of it (or, of course, if you await its result), you're blocking the outer function. The point of async do (and of this thread's spawn) is that you can await without blocking the outer function.

I see, so await do { } would solve an entirely unrelated issue. Thanks for the clarification.