counterfeit-monkey

counterfeit-monkey copied to clipboard

For Kerkerkruip we run some automated tests on Travis: https://travis-ci.org/i7/kerkerkruip/builds/53228540

Is this something that could be useful to set up for CM? What kind of regressions would we like to check against? Just using the built in tests?

Simply running TEST ME, TEST ME2 and TEST HARD and then reporting any differences in the transcript would be a great start.

Of course it would be amazing to have a way to tell if a code change improves performance or slows it down. Would the time taken to run the full-game tests be a useful measure of this?

Of course it would be amazing to have a way to tell if a code change improves performance or slows it down. Would the time taken to run the full-game tests be a useful measure of this?

Yeah that would make sense - it would be a lot better than micro benchmarks too. Travis works for pull requests, so if there's a controversial change and we had it doing a profile run as a matter of course, then that would make it really easy to see the effect of the pull request.