dramatiq
dramatiq copied to clipboard
Feature Request: iterate over a group as actors complete
concurrent.futures
(and probably other Python stdlib stuff) has the idea of iterating over results from concurrent processing as they are completed:
https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.as_completed
I'm currently using the Group.get_results()
method to iterate over a group of results, but it would be nice to be able to do that as the results come back instead of in the order they were queued. That way, if I have a network request hang at the very beginning, I can get other work done while that action times out and is retried.
I don't think there's a way for us to do this in a low-overhead way, but I'd be happy to be proven wrong if someone is willing to take this up.
I'm willing to experiment with this, I have an idea how to pull it off.
@synweap15 go for it!