newspaper
newspaper copied to clipboard
Convert mthreading to use concurrent.futures
to provide support for futures and async processing
This came about when I wanted to be able to process jobs as they were completed, but the only supported operations for the thread pool was to submit new jobs, and to wait for all the jobs to complete. This left the main thread blocked unnecessarily.
Thanks so much for this change @c0d3d, will take some time to review 👍 💯
Thanks for the feedback. I'll make the requested changes. As far as throughput metrics go I have none, but I can throw some together. I'll probably add some as a test to the repo that can be used as a baseline in the future, that way people can just run them and see.
heads up https://github.com/codelucas/newspaper/pull/618, which cleans up some dead code and adds a thread count override to NewsPool
is merging, may be some small merge conflicts
Is there any update on this merge request? I would love to use this functionality!
@hugocool I never got around to making the requested changes or benchmarks, so now progress.