Scala 3 / Dotty plans?

[halfninja]

Just wondering if anybody had taken a look at how/if scala-async might work in Scala 3, given that the Scala 2 macro system as we know it will not survive? Is it still too soon to say?

It would be good to get some vibes about how confident we can be that the library might survive in some form into Scala 3, for deciding whether it's worth adopting it in an application.

[retronym]

It will find its way to Scala 3 in one form or another without requiring user code to change.

We're actually experimenting at the moment with making this an actual compiler phase of Scalac, given how important the use case is. (Many other language that have async/await support have it baked into the compiler). So if we go down that path, it would also become a phase of the Scala 3 compiler.

[mushtaq]

@retronym , if it becomes a phase of Scala3 compiler, will it be possible for a method (not value) to return a T instead of Future[T] by using a top level await? Also will it be possible to use await inside a lambda?

I really love how Kotlin coroutines allow you to do the above given that it captures the suspend effect using keyword.

[SethTisue]

https://github.com/rssh/dotty-cps-async exists, not sure what its future is

[retronym]

@mushtaq Moving into the compiler doesn't change the main restrictions on placement of awaits. await in pattern guards and Boolean.{||, &&} is the only incremental improvements in this area coming in async 1.0.

Try/catch support (#89) is doable.

Supporting use cases that await in lambdas is a bit harder. We could do some ad-hoc inlining of foreach to while loops over iterators expand the horizon of valid await. Similarly we could inline Option combinators and some collections combinators.

A more general solution requires a library of continuation-lifted versions of the combinators (#32) eg:

def List_map_lifted[A, B](self: List[A])(f: A => Future[B]): Future[List[B] = async {
    val result = ListBuilder[B]()
    var rest = self
    while (!rest.isEmpty) {
      result += await(f(rest.head)
      rest = rest.tail
    }
   result.result()
}

And then a translation step to convert:

def userCode = async {
   List(1, 2 , 3).map(x => x * await(otherFuture))
}

To:

def userCode = async {
   List_map_lifted(List(1, 2 , 3)(x => async { x * await(otherFuture) })
}

Which would be valid for async.

A slight variant of the them is to require the user to mark the spots where a rewrite is needed:

def userCode = async {
   List(1, 2 , 3).asyncably.map(x => x * await(otherFuture))
}

Where asyncably provides the extension method map as a macro that does the rewrite to call.

The user could also opt in to running the map in parallel if they know that the lambda is pure.

[SethTisue]

"I expect this will be ported to Scala 3", says Martin O: https://twitter.com/odersky/status/1276577949374955521

[rssh]

btw, https://github.com/rssh/shim--scala-async--dotty-cps-async now can be a workaround. (At least, all current test-cases are passed).

[yanghao]

@mushtaq Moving into the compiler doesn't change the main restrictions on placement of awaits. await in pattern guards and Boolean.{||, &&} is the only incremental improvements in this area coming in async 1.0.

Try/catch support (#89) is doable.

Supporting use cases that await in lambdas is a bit harder. We could do some ad-hoc inlining of foreach to while loops over iterators expand the horizon of valid await. Similarly we could inline Option combinators and some collections combinators.

A more general solution requires a library of continuation-lifted versions of the combinators (#32) eg:

def List_map_lifted[A, B](self: List[A])(f: A => Future[B]): Future[List[B] = async {
    val result = ListBuilder[B]()
    var rest = self
    while (!rest.isEmpty) {
      result += await(f(rest.head)
      rest = rest.tail
    }
   result.result()
}

And then a translation step to convert:

def userCode = async {
   List(1, 2 , 3).map(x => x * await(otherFuture))
}

To:

def userCode = async {
   List_map_lifted(List(1, 2 , 3)(x => async { x * await(otherFuture) })
}

Which would be valid for async.

A slight variant of the them is to require the user to mark the spots where a rewrite is needed:

def userCode = async {
   List(1, 2 , 3).asyncably.map(x => x * await(otherFuture))
}

Where asyncably provides the extension method map as a macro that does the rewrite to call.

The user could also opt in to running the map in parallel if they know that the lambda is pure.

I am new to Scala ... and find it is very difficult with async/await ... any idea if dsl.scala (https://github.com/ThoughtWorksInc/Dsl.scala) poses no restriction on where await can appear, or it is not that different? Thanks.

[SethTisue]

@yangbao I'd suggest asking on https://users.scala-lang.org

[mushtaq]

Now that Scala 3 is out, is there any decision on porting this lib? @retronym

[retronym]

It's definitely something I want to work on, but I need to find a block of time to do the port. Hopefully will happen mid year.

[mushtaq]

Good to know that @retronym! This will help a lot of projects to upgrade to Scala 3.

[mushtaq]

A gentle reminder that someone is eagerly awaiting the Scala 3 port :)

[rssh]

Btw, can I ask - why dotty-cps-async or shim--scala-async--dotty-cps-async (which is out of the box substitution) does not work for you? [not sure that this is the right place for this question, feel free to remove this or move to discussions]

[mushtaq]

@rssh have not tried it yet.

async/await is spread across our code base. If shim--scala-async--dotty-cps-async gets official blessing from scala-center, it will make it easier.

I am sure it works, but a bit hesitant to put in the releases.

[Atry]

I just wanted to let you know I also ported Dsl.scala to Scala 3, and provided a Scala Async polyfill as well.

libraryDependencies += "com.thoughtworks.dsl" %%% "scala-async" % "latest.release"

[fdietze]

Just wanted to say that https://rssh.github.io/dotty-cps-async/index.html is working very well for me, so far.

[SethTisue]

relevant new pre-SIP from 47 Degrees: https://contributors.scala-lang.org/t/pre-sip-suspended-functions-and-continuations/5801

[SethTisue]

and another new player in this space, from Prof. Odersky himself: https://github.com/lampepfl/async — includes slide deck from his talk at Scalar last week