Setting max degree of concurrency for multiple tasks

[mariusz96]

Help us make content visible

Hi,

Recently, there came a client's (or maybe devs' actually? lol) requirement:

• to process (thousands of) data records individually with a method which includes sending http request to an external web api for each individual item but don't start all processings at once and instead limit concurrency based on some variable in the configuration .

That seemed fairly basic but I could not find any simple example of doing this in the docs (I like particularly these ones as they do not use Task.Run / TaskFactory.StartNew and other advanced / uncommon System.Threading APIs).

Describe the new article

Eventually, we settled on a solution similar to this:

var data = Enumerable.Range(1, 27);
int maxDegreeOfConcurrency = 10;

var dataChunks = data.Chunk(maxDegreeOfConcurrency);

foreach (var dataChunk in dataChunks)
{
    await Task.WhenAll(dataChunk.Select(dataItem => DoSomethingAsync(dataItem)));

    Console.WriteLine();
    await Task.Delay(4000);
}

async Task DoSomethingAsync(int i)
{
    Console.WriteLine($"in: {i}");
    await Task.Delay(1000);
    Console.WriteLine($"out: {i}");
}

but I would like a simple, "recommended" way of solving the above problem somewhere in the docs.

Also, a basic article with guidance on Parallel.ForEachAsync vs Task.WhenAll would be appreciated.