python-pool-performance icon indicating copy to clipboard operation
python-pool-performance copied to clipboard

Interesting results

Open anentropic opened this issue 6 years ago • 2 comments

These results are interesting because the conventional wisdom is that multiprocessing is good for CPU-bound and greenlets or threads are good for IO-bound.

Yet in these tests the greenlet libs win the CPU-bound and multiprocess wins the IO-bound.

I wondered if maybe the CPU-bound test itself was too brief. In other words, although the body of the job is doing only CPU and no IO, the actual task is too trivial so that serialization overhead (of args and results) rather than 'work' dominates in the multiprocess case.

Obviously in many situations this is a realistic CPU-bound workload and the result is still interesting.

anentropic avatar Oct 26 '18 10:10 anentropic

@anentropic Thanks for your feedback! That's a good point, do you have any suggestions for a better CPU-bound test? If I remember correctly, sqrt increased the CPU the most in my testing, but I'm sure there's room for improvement.

JohnStarich avatar Oct 31 '18 03:10 JohnStarich

I think you could still use sqrt for example, but just do it like 100 or 1000 times in a loop within the function under test. Then the individual cpu-bound jobs would take longer. Might be interesting as a separate test, see how it affects the results.

anentropic avatar Oct 31 '18 07:10 anentropic