r7rs-benchmarks
r7rs-benchmarks copied to clipboard
CPU limit penalizes parallel GC
The CPU limit in the bench script is 300 seconds of CPU time (5 minutes). Compilation has 5 minutes, then running has 5 minutes. But unfortunately this penalizes parallelization, whose goal is usually to minimize wall-clock time rather than CPU time; and the end result of the benchmarking process is wall-clock time. Of course this result is worse the more cores you have.
I think Guile has hit this limit in ctak at run-time, and now that some optimizations allow for better visibility into compile, we hit it there at compile-time as well.
I think the limit should be raised to 15 minutes. It's a decent trade-off between the concern of wanting short benchmark runs and allowing for parallelism in GC.
Does anyone know whether there's a simple way to limit wall-clock time, not user time?
could https://linux.die.net/man/1/timeout help here? (probably not going to be entirely as precise as ulimit but may work well enough)