dramatiq icon indicating copy to clipboard operation
dramatiq copied to clipboard

Feature Request: iterate over a group as actors complete

Open rsyring opened this issue 4 years ago • 3 comments

concurrent.futures (and probably other Python stdlib stuff) has the idea of iterating over results from concurrent processing as they are completed:

https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.as_completed

I'm currently using the Group.get_results() method to iterate over a group of results, but it would be nice to be able to do that as the results come back instead of in the order they were queued. That way, if I have a network request hang at the very beginning, I can get other work done while that action times out and is retried.

rsyring avatar Feb 27 '20 20:02 rsyring

I don't think there's a way for us to do this in a low-overhead way, but I'd be happy to be proven wrong if someone is willing to take this up.

Bogdanp avatar Mar 06 '20 08:03 Bogdanp

I'm willing to experiment with this, I have an idea how to pull it off.

synweap15 avatar Mar 06 '20 20:03 synweap15

@synweap15 go for it!

Bogdanp avatar Mar 07 '20 08:03 Bogdanp