Automation & Tracking

[rwjblue]

I would like to be able to use this project to run these tests automatically and track performance over time using the latest Ember builds. This obviously will be somewhat difficult as they are moving quite quickly right now, but the API changes at least should settle down after 1.0.0 (hopefully 2013/08/31).

Ultimately, I would love to see this sort of performance regression testing incorporated with the standard ember test suite or at least have the ember-dev tooling kick off a run on commit.

If you are open to this I think the first few major steps are:

• [ ] Setup automated test runner. Probably using Karma so that we can test on various browsers.
• [ ] Setup an API server to receive results from Perf.Profiler. Grab at least the following data elements:
  • [ ] Ember version (preferably the commit and version number)
  • [ ] Browser type & version
  • [ ] Test execution time
  • [ ] ???
• [ ] Make new tests easier to implement. Perhaps use separate template files that are then concatenated at 'build'?
• [ ] Precompile the handlebars templates? Not sure if currently the compilation is included in the test run time (probably not), but would want to confirm.

I would like to start working on this soon, and really just want to make sure you agree with the direction before I begin with this repo as a starting point

[rwjblue]

@eviltrout - I would also love to know your general thoughts on this.

[eviltrout]

This is an awesome idea and I've love to see it go in that direction. Let me know if I can do anything to help!