relude icon indicating copy to clipboard operation
relude copied to clipboard

Array.IO.sequence vs Promise.all/Promise.any

Open BlueHotDog opened this issue 3 years ago • 6 comments

Hey, trying to understand how Array.IO.sequence is compared to Promise.all/Promise.any. Are the IOs running on parallel? Is it like Promise.all that if one fails all fails? (looks like, since it doesnt look like it collects the errors) Maybe worth having an Promise.any solution too?

BlueHotDog avatar May 13 '21 12:05 BlueHotDog

It runs them in parallel at present. I'd have to check, but I believe it preserves order, and emits the first error occurred (based on the order of the array, not of resolution), not all errors.

AlexaDeWit avatar May 13 '21 15:05 AlexaDeWit

Yeah, I think @AlexaDeWit is right - if I recall it runs the async work in parallel (because the default Applicative for IO is setup to run the tasks in parallel), and it has fail-fast semantics, where it bails as soon as there is a single error and returns that error. It doesn't have error-collecting semantics like something like Validation, but that would be nice to have.

I don't think we have an any kind of function right now, but that would also be nice to have - this is sometimes called race. If anyone wants to PR that change, that would be cool. I haven't had a lot of time to dedicate to relude recently, but I can try to add it at some point.

andywhite37 avatar May 13 '21 18:05 andywhite37

I just talked to @BlueHotDog a bit in discord and wanted to bring a few things into this thread:

  • IO.all is most definitely parallel, we confirmed this
  • To get any like behavior, you can do:
module IOWithVoidError =
  IO.WithError({
    type t = Void.t;
  });

let foo = iosToRun => 
  iosToRun
  |> List.map(IO.summonError)
  |> IOWithVoidError.all
  |> IO.map(res => Js.log2("done", res))
  |> IO.mapError(ignore);

The key here using IO.summonError and making the IOWithVoidError module

johnhaley81 avatar May 13 '21 18:05 johnhaley81

thanks @johnhaley81 ! was just about to write that :)

Since i'm using rescript, the code i ended up with is:

module IOWithVoidError =
  IO.WithError({
    type t = Void.t;
  });

let collect = array => {
  array
  |> List.fromArray
  |> List.map(IO.summonError)
  |> IOWithVoidError.all
  |> IO.map(res => res |> Relude.Array.fromList)
  |> IO.mapError(ignore);
};

It would be cool to have allArray or something to avoid the array->list->array conversion

BlueHotDog avatar May 13 '21 18:05 BlueHotDog

While we're talking about IO, I would just put it out there that the implementation is probably not the best. IO is not stack-safe and is over-complicated in terms of all the constructors. I think if I could go back, I probably would have greatly simplified it. I like the current interface of IO, but the underlying implementation could probably be reworked a bit, if that would help with the addition of new features.

Making it stack-safe would be a really nice-to-have. It might also make sense to just create a new "async effect type" off to the side of IO (in relude or in a separate library), prove it out, and then eventually replace IO with this new type. That would be a bigger effort if anyone is interested.

Desired new features:

  • stack safety
  • maybe the option of having different Applicative/Monad instances (one for fail-fast and one for parallel/error-collecting)
  • simplification of the constructors
  • other things?

andywhite37 avatar May 13 '21 18:05 andywhite37

Yeah, I think @AlexaDeWit is right - if I recall it runs the async work in parallel (because the default Applicative for IO is setup to run the tasks in parallel), and it has fail-fast semantics, where it bails as soon as there is a single error and returns that error. It doesn't have error-collecting semantics like something like Validation, but that would be nice to have.

I don't think we have an any kind of function right now, but that would also be nice to have - this is sometimes called race. If anyone wants to PR that change, that would be cool. I haven't had a lot of time to dedicate to relude recently, but I can try to add it at some point.

If you're happy with what I'm suggesting in #298 I'd be happy to do both simultaniously?

AlexaDeWit avatar May 13 '21 23:05 AlexaDeWit