FSharp.Control.AsyncSeq icon indicating copy to clipboard operation
FSharp.Control.AsyncSeq copied to clipboard

Proposing AsyncSeq.ofSeqAsync and AsyncSeq.concat

Open blumu opened this issue 7 years ago • 3 comments

Description

Would the maintainers consider including the following two helpers:

  • We already have AsyncSeq.init and AsyncSeq.unfoldAsync to generate new AsyncSeq, but I could not find any helper to create one from an existing sequence of asynchronous computations (seq<Async<_>>). The following helper ofSeqAsync would help with that:
module AsyncSeq =
    let ofSeqAsync (s:seq<Async<'t>>) : AsyncSeq<'t> =
        asyncSeq {
            for asyncElement in s do
                let! v =  asyncElement
                yield v
        }
  • Using AsyncSeq.concatSeq, it's currently possible to concatenate sequences of AsyncSeq (as in seq<AsyncSeq<_>>) but not an AsyncSeq of AsyncSeqs. I propose adding the following function.
module AsyncSeq =
    let concat (s:AsyncSeq<AsyncSeq<'t>>) : AsyncSeq<'t> =
        asyncSeq {
            for innerSeq in s do
                for e in innerSeq do
                    yield e
        }

Example use case

Suppose you are given:

  • enumerateFiles : string -> Async<AsyncSeq<string>> that asynchronously returns an asynchronous enumeration of all files under a given directory;
  • and enumerateDirectories : string -> AsyncSeq<string> that asynchronously enumerates all sub-directories under a given directory;

and you want to enumerate all files under all directories under c:\ then with the two helpers you could write it as follows:

enumerateDirectories "c:\"       // : Async<AsyncSeq<string>>
|> Seq.map enumerateFiles       // : seq<Async<AsyncSeq<string>>>
|> AsyncSeq.ofSeqAsync          // : AsyncSeq<AsyncSeq<string>>
|> AsyncSeq.concat              // : AsyncSeq<string> 

I undertand that the function AsyncSeq.mergeAll could be used to achieve the same goal, except that it would require first enumerating all directories and convert them into a list, before doing the concatenation.

Or perhaps there is a better way to do this?

blumu avatar Dec 29 '18 08:12 blumu

Yes, these are both sensible design additions

dsyme avatar Jan 08 '19 19:01 dsyme

Yes, I think all of these make sense, PR would be great :)

eulerfx avatar Jan 09 '19 13:01 eulerfx

@eulerfx @dsyme I've submitted PR #102 to add those two helpers. Let me know if the unit tests meet your expectations or if you think more is needed.

blumu avatar Jan 13 '19 09:01 blumu