otp icon indicating copy to clipboard operation
otp copied to clipboard

Feature request: Function equivalent to elixir's `Task.async_stream`

Open TanklesXL opened this issue 4 years ago • 7 comments

As previously discussed, this would be a nice & useful function to have.

TanklesXL avatar Jan 12 '22 23:01 TanklesXL

Great suggestion. Initial thoughts:

The name isn't very clear- what would be clearer?

The Elixir function is lazy, which makes me think it spawns a process per element requested in order to avoid leaking processes if you never reach the end. Would a version that always processes the entire collection but uses a work pool be advantageous?

lpil avatar Jan 13 '22 17:01 lpil

In keeping with the original intent of async_stream maybe a name like concurrent_map, parallel_map? Might be nice to have an alternative that works on a list of thunks as well maybe?

TanklesXL avatar Jan 13 '22 19:01 TanklesXL

parallel_map sounds nice and descriptive. What do you mean about thunks?

lpil avatar Jan 16 '22 16:01 lpil

With regards to thunks I was mostly looking for a way to run a list of fn() -> a concurrently without needing to do something like

l
|> list.map(task.async)
|> list.map(task.await_*...)

TanklesXL avatar Jan 16 '22 18:01 TanklesXL

Maybe an api like ocaml's parmap lib might be nice?

https://rdicosmo.github.io/parmap/doc/parmap/Parmap/index.html

TanklesXL avatar Jan 16 '22 18:01 TanklesXL

I'm not sure how that would translate into Gleam as we don't have module functors. Keen to see what you have in mind

lpil avatar Jan 16 '22 19:01 lpil

Oh that's a good point, hmm this definitely requires more thought.

Might be worth opening a discussion around this issue? I'd be interested in seeing if anyone has ideas.

My link to parmap was originally for inspiration in the sense of what useful functions might be wanted and things like being able to parametrize by desired number of workers and such, not really for the underlying implementation

TanklesXL avatar Jan 16 '22 20:01 TanklesXL