Immediate queues

[joelving]

First off, thank you for a wonderful project! It has really saved me a ton of time rolling my own.

As I'm reading the code, it seems I cannot avoid waiting up to 1 second before jobs being dispatched, since the timer cannot be supplied a shorter timespan. I realize this may not be a show-stopper (sure isn't for me), but I was wondering why the queues adopt a polling approach instead of something like that mentioned in the docs. That way, items are dispatched as soon as possible (unless stopped by https://github.com/jamesmh/coravel/issues/140) without incessant polling.

There may be considerations I'm unaware of, so forgive me, if it's obviously infeasible or undesirable.

[jamesmh]

Thanks!

That's an interesting idea. I decided to go with the polling approach just to give more control over how often batches of jobs will execute.

That being said, right now the queue is using one thread asynchronously. I'd like to change that and (a) have multiple logical queues and (b) allow opt-in multiple thread processing.

I'd like to build a distributed queue at some point too. In that case, control over the time-lapse between producing and consuming might not be as granular 🤷‍♂️.

I feel like those feature would be more beneficial?

Thoughts 😄?

[joelving]

I'm unsure whether await Task.WhenAll(dequeuedTasks.Select(InvokeTask).ToArray()); (which you're currently doing) would lead to them being processed on different threads by the thread pool. I suspect it does, which means you're already multi-threaded (though on the thread pool, which I think is fine).

I've used the approach from the docs in combination with not-awaited Task.Runs to ensure that the QueuedHostedService is only used to dispatch the jobs to the thread pool, giving it a pretty high throughput. See this woefully out-of-date piece of code for an example.

As for logical queues, I don't know what makes most sense across all the use cases you'd want to support. I suspect you can get fair with generics or maybe an approach similar to how they do named http clients?

More generally, I think the strength of Coravel is it's simplicity, so I'd be careful making it more complicated than necessary - but that's just my personal opinion.

[jamesmh]

WhenAll would process the asynchronous I/O in concurrently, but any CPU bound work will "stall" the other Tasks from continuing their own CPU bound work. My idea would be to run individual jobs on their own thread completely (but throttle the total number of "active" threads). Anyways...😅

I like what you had to say:

I think the strength of Coravel is it's simplicity, so I'd be careful making it more complicated than necessary.

I agree. And yes, it's tough to know where the fine line is. Especially in terms of improving scalability and adding some distributed features. I appreciate you understand that simply adding more features is sometimes not the most beneficial thing 😂

For now, I'd say putting the time-lapse to 1s would be the best for now until I look at improving the queue overall.

[joelving]

WhenAll would process the asynchronous I/O in concurrently, but any CPU bound work will "stall" the other Tasks from continuing their own CPU bound work. It depends on how you dispatch the tasks.

I got insecure and then curious 😂, so I made a POC-gist, which illustrates the behavior. Note that the jobs are simply incrementing a counter, so they're entirely CPU-bound.

When using Task.Run, the tasks are scheduled on the thread pool which means they are run on whichever threads are available - 10 distinct threads instead of 1 in this example. Being the thread pool, it will scale according to need, so it's not impossible - though not recommended!

A way to do it would be to provide different overloads leading to different methods of scheduling the tasks.

• Code that only blocks for a short while at a time on the thread pool using Task.Run
• Code that blocks for a long time on separate threads using Task.Factory.StartNew

This is seriously tricky territory, so I'd consult with David Fowler or similar expert before committing to an approach - but I'm pretty sure you can get multi-threading fairly easily. 😃