perf_check issues

Extract summary statistics from test_case.rb

Create one single class to compute summary statistics on a list of profiles.

Gather profiles globally for the entire run

An object representing a run or `perf_check` could keep all the profiles. Then `perf_check.profiles` could return all the measurements. Statistics code and figure out how to interpret depending on the...

Manfred

Add authentication support

2

Most applications use some form of authentication so it's probably useful to add some sort of mechanism to PerfCheck to select a role and/or user for authentication. I think we...

Manfred

Perf check should exit early when important command fails

There is no really good reason to continue trying to perform requests when a git reset fails. We could technically differentiate between fatal errors (e.g. git issues) and acceptable failures...

Manfred

Manage `tmp/caching-dev.txt` for Rails 5+

sudara

delay perf_check runs until the server is actually ready for service

putting '@PerfCheck' into a github PR causes a perf check to be run. However, when the database that perf check uses is being backed up, perf-check reports spurious, random errors....

aks

support envar changes as part of testing

Some app changes appear only with envar changes, so it should be possible to set envars on the "control" test and set the same envars on the "test" runs. Suggested:...

aks

Evaluate cache_classes effect on request times

Let's check out whether it's a need to force-set cache_classes when code isn't being changed between requests. Basically, is it a consistent amount of overhead? Widely variable?

sudara