devtools
devtools copied to clipboard
flutter
remove unneeded delays in test execution
The PR (https://github.com/flutter/devtools/pull/93) to deflake the integration tests introduced several delays, not all of which are likely necessary. We should remove the ones we find aren't necessary - this will recapture 1-2 seconds of test execution time per test.
Note: the position of some of the delays was moved to accommodate the windows CI (details here: https://github.com/flutter/devtools/pull/218).
The delay I added in #218 was removed (we now have an app.devToolsReady event), though I don't think I've removed any others (I think we still have a lot). I'm not sure what the effort would be to remove them, though it would be a nice change (currently we have to trade off making the tests too long versus waiting long enough to be sure the test will work - whereas if we can change them to wait on something specific, we could increase the delays and reduce flakiness - if we have any).
Maybe we should try removing a couple each week and try to fix up any tests that become flaky (and if we need to put the delay back, add a comment so we know we've tried that one). Alternatively, we could erode how long the delay is slowly, doing the same.
(Again, not sure how good the trade-off is between effort/benefit, but I've been working on devtools less lately so haven't had to wait for builds much!).
FWIW, I reduced the delays from 100 and 500ms to 1ms and 5ms on a branch, and only 2 tests per-bot failed:
https://travis-ci.org/DanTup/devtools/builds/562960623
It's possible it's made others more flaky, but Windows runs super slow, and even that only had 1-2 failures per bot so maybe getting rid of most of them wouldn't be too tough.