pyperformance
pyperformance copied to clipboard
Add benchmark for IPC
AFAIK, there is no benchmark between process communication. It looks good to add such benchmarks if possible.
This is a great contribution. I just want to flag that this is potentially more of a benchmark of I/O and OS system level performance than it is of the Python interpreter. Still fine to include it, but we may want to somehow isolate it from the other benchmarks that are primarily CPU-bound. But I'm happy to merge this and collect some data first.
I just want to flag that this is potentially more of a benchmark of I/O and OS system level performance than it is of the Python interpreter.
Yeah that's what I intended
@ericsnowcurrently @markshannon I would like to merge this benchmark if there is no serious concern, please let me know if you have any concern :)
We might want something a bit more "macro" then this. Maybe something like:
import multiprocessing
def f(x: int) -> int:
return x
def bench_multiprocessing_pool(p: int, n: int, c: int) -> None:
with multiprocessing.Pool(p) as pool:
for _ in pool.imap(p, range(n), chunksize=c):
pass
That way we cover more than just the raw byte transmission: the multiprocessing machinery, interpreter startup/shutdown, import/unmarshalling, pickling/unpickling, etc. Thoughts?
That way we cover more than just the raw byte transmission
Yeah, that looks nice, I will update the benchmark with such variations.
@brandtbucher PTAL