RFC 232 tests are slower

[stopdropandrew]

I recently upgraded an app to Ember 3.1.4 (from 2.1.8), we also switched to RFC 232 style tests as well (great improvement!). But, we noticed that our test suite run length has more than doubled.

At first I thought we were doing something strange in our app causing the issue, but after a little exploration noticed that it seems to be an inherent issue with the new helpers. Ember twiddle doesn't support the new style tests yet, but I set up an example app, using tests generated from 3.1.4 (RFC 232) and 2.1.8 (moduleFor), and a few just asserting true.

https://github.com/stopdropandrew/ember-rfc232-slow-tests

Here's an example of a typical test run:

[Turbo87]

as mentioned on Slack, part of it is that instance initializers now run for every test (previously only acceptance tests), which can cause a slight slowdown. but you are right that the test helpers also seems to be slower, which appears to be caused by them doing a lot more things asynchronously. I expect this to get better over time though, while the work on backburner.js is progressing.

[cibernox]

FWIW, my test suite in particular actually speeded up a bit with the new test approach. If your tests are significantly slower I'd check if your instance initializers are doing something particularly slow, as that is the main difference.

[Turbo87]

@cibernox if you look at the repro repo the initializers don't seem to be the problem. Ember Data is causing a bit of startup slowness, but even with it removed the tests are still significantly slower.

I would assume though that once the acceptance tests have more content than just a single assertion the speed difference shouldn't be that high anymore.