newspaper icon indicating copy to clipboard operation
newspaper copied to clipboard

Convert mthreading to use concurrent.futures

Open c0d3d opened this issue 6 years ago • 5 comments

to provide support for futures and async processing

This came about when I wanted to be able to process jobs as they were completed, but the only supported operations for the thread pool was to submit new jobs, and to wait for all the jobs to complete. This left the main thread blocked unnecessarily.

c0d3d avatar Apr 17 '18 18:04 c0d3d

Thanks so much for this change @c0d3d, will take some time to review 👍 💯

codelucas avatar Aug 26 '18 18:08 codelucas

Thanks for the feedback. I'll make the requested changes. As far as throughput metrics go I have none, but I can throw some together. I'll probably add some as a test to the repo that can be used as a baseline in the future, that way people can just run them and see.

c0d3d avatar Sep 02 '18 17:09 c0d3d

heads up https://github.com/codelucas/newspaper/pull/618, which cleans up some dead code and adds a thread count override to NewsPool is merging, may be some small merge conflicts

codelucas avatar Sep 03 '18 06:09 codelucas

Is there any update on this merge request? I would love to use this functionality!

hugocool avatar Aug 18 '21 15:08 hugocool

@hugocool I never got around to making the requested changes or benchmarks, so now progress.

c0d3d avatar Aug 18 '21 23:08 c0d3d