AsyncEnumerable icon indicating copy to clipboard operation
AsyncEnumerable copied to clipboard

Using ParallelForEachAsync() for non-CPU-intensive tasks

Open DumboJetEngine opened this issue 3 years ago • 0 comments

I would like to use a parallel loop for N number of calls to a service, to execute them in batches, and avoid DoS-ing the service. I had previously created my own version of the loop, using Task.WhenAny() and Task.WhenAll(), but then I needed that in multiple places and thought I should find myself a library for it (unfortunately, I don't use .NET 6 yet, so I don't have its features).

Looking at your ParallelForEachAsync method: https://github.com/Dasync/AsyncEnumerable/blob/master/src/Extensions/ParallelForEachExtensions.cs#L279 I see that it uses Task.Run() which would be using a new thread. But I guess that this could be avoided in my case, couldn't it? So, in other words, your library is not optimized for my use-case, correct? Or perhaps this code is only used to prevent deadlocks? Let me know if I am missing anything. :)

I wonder if the Parallel.ForEachAsync() method in .NET 6 works the same way as your library: https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.parallel.foreachasync?view=net-6.0

DumboJetEngine avatar Apr 21 '22 08:04 DumboJetEngine