benchy
benchy copied to clipboard
treeform
Allow configuring runtime cap - 60 seconds might be too long in some cases
I decided to add this package on playground (https://github.com/PMunch/nim-playground/pull/7), but because maximum execution time is capped at 20s it might be possible for the benchmark to be terminated externally without even finishing.
I know that 60s cap is not always reached, but considering playground has relatively low execution power it might happen more often compared to regular use.
at least 10 times but possibly more to figure out the standard deviation. It will keep running it until things look like they stabilized. It will stop after 60s though.
I thought about adding cappedTimeIt(maxSeconds: int, name: string, body: untyped) that would allow configuring maximum running time. Make timeIt an alias for cappedTimeIt(60, ...)
Does the implementation idea sound good, or you would prefer to keep this package as simple as possible, without any configuration complexity?
Sorry I don't understand you want it to be infinite? or 20seconds?
The time cap is there really as a fail safe. Most benches should finish in 1-5s seconds. No one has time to wait a full minute!
You can already pass it max number of iterations: https://github.com/treeform/benchy/blob/master/tests/test.nim#L52
I can see another param that passes max time too.
Sorry I don't understand you want it to be infinite? or 20seconds?
Just as the title says - I want to be configurable. I didn't know you can pass the maximum number of iterations (it is not mentioned in README, and I didn't look into tests (although I should've done this before opening an issue)). I think that solves an issue, though an option to have "another param that passes max time too." would be exactly what I need in this case.
The time cap is there really as a fail safe. Most benches should finish in 1-5s seconds. No one has time to wait a full minute!
Yes, of course, but in https://github.com/PMunch/nim-playground/pull/7 PMunch was concerned with the possibility of the benchmark executing for longer than twenty seconds.
Oh, to be honest I thought that all the benchmarks ran for a minute. If it only runs for a couple seconds it should be fine to run on the playground.
