python-pool-performance
python-pool-performance copied to clipboard
Interesting results
These results are interesting because the conventional wisdom is that multiprocessing is good for CPU-bound and greenlets or threads are good for IO-bound.
Yet in these tests the greenlet libs win the CPU-bound and multiprocess wins the IO-bound.
I wondered if maybe the CPU-bound test itself was too brief. In other words, although the body of the job is doing only CPU and no IO, the actual task is too trivial so that serialization overhead (of args and results) rather than 'work' dominates in the multiprocess case.
Obviously in many situations this is a realistic CPU-bound workload and the result is still interesting.
@anentropic Thanks for your feedback! That's a good point, do you have any suggestions for a better CPU-bound test? If I remember correctly, sqrt
increased the CPU the most in my testing, but I'm sure there's room for improvement.
I think you could still use sqrt
for example, but just do it like 100 or 1000 times in a loop within the function under test. Then the individual cpu-bound jobs would take longer. Might be interesting as a separate test, see how it affects the results.