php-dependency-injection-benchmarks
php-dependency-injection-benchmarks copied to clipboard
TRPB
use tavis-ci to run the benchmarks
Does that make sense?
I mean, if you run the benchmarks on a system where other things may be starting and stopping all the time, using simple microtime(true) calls is going to produce wildly inconsistent results, isn't it?
Ideally, it needs to run on a system under no load, or at least under constant, continuous load.
Personally I'd prefer to use a simple self contained benchmark script. The one I supplied does 10 runs of each tests and takes an average I'm not sure whether travis would give any more of an accurate result.
The one I supplied does 10 runs of each tests and takes an average
Maybe take a look at the fairly simple benchmark thing I wrote.
Averages are not a very reliable way to benchmark. If you run the scripts multiple times, you can see why - deviations can be quite large.
My benchmark thing uses weighted averages, which is a statistical way to "correct" for "error" in some of the readings - then takes a set of samples and runs weighted average on those, which improves precision and gives readings that are pretty close to the same every time. Or at least a lot close than averages. Worth a look maybe, it'll take like 2 minutes to set up ;-)
I'll have a look, if you check https://github.com/TomBZombie/php-dependency-injection-benchmarks/blob/master/test1-5_runner.php you'll see it ignores any results > 120% of the smallest result to stop any wild deviations :)
I've also been running it on a desktop under no load, rather than an active webserver.