flutter icon indicating copy to clipboard operation
flutter copied to clipboard

Fix performance problems of `flutter test --experimental-faster-testing`

Open derekxu16 opened this issue 2 years ago • 3 comments

The performance of flutter test --experimental-faster-testing is currently not ideal when running a large number of test files because the Lightweight Engines spawned when using this strategy get blocked from shutting down by open ReceivePorts. This was my latest attempt to ensure that all channels, and thus all ReceivePorts, get closed on the spawned isolates: https://github.com/flutter/flutter/commit/d560cf4ac29ded2198c022a0bfbdab6fd1a43e17, but this commit did not fix the problem. So, we either have to determine what other changes need to be made in order to ensure that all ReceivePorts get closed on spawned isolates, or alternatively we need to add a new Engine API that can forcibly kill spawned Lightweight Engines.

derekxu16 avatar Apr 10 '24 21:04 derekxu16

cc @a-siva

derekxu16 avatar Apr 10 '24 21:04 derekxu16

//cc @jonahwilliams would you be the right person to help out Derek here ?

a-siva avatar Apr 10 '24 21:04 a-siva

@derekxu16 are these package:test send/receive ports? if so have you talked to any of the package:test maintainers yet? I can look at it a bit but I'm not very familiar with the work.

jonahwilliams avatar Apr 10 '24 22:04 jonahwilliams

There was some discussion with the package:test maintainers here: https://chat.google.com/room/AAAA3-KvHNw/_Ng7rX3hbHk. Nate's last message indicated that package:test currently forcibly kills spawned isolates when they are done running test code, and he doesn't know whether or not the spawned isolates would shut down by themselves if the force-killing logic was removed. When I get the chance, I can check what happens if the force-killing logic is removed from package:test.

derekxu16 avatar Apr 11 '24 18:04 derekxu16

and he doesn't know whether or not the spawned isolates would shut down by themselves if the force-killing logic was removed.

They do in the typical case.

After some more thinking, though, I got worried about removing the kill() call. I think it might be in place as a resource optimization. Nothing breaks when I remove it - but if some test patterns are commonly holding resources which keep test isolates running don't know if it could become a noticeable performance issue.

https://github.com/dart-lang/test/pull/2198#issuecomment-2015862267

natebosch avatar May 02 '24 18:05 natebosch

The culprit of slow performance seems to be the fact that all dart code for all tests in one combined dill file for all concurrently-running dart isolates spawned in flutter_tester runs on a single one and the same platform thread. So effectively we end up running all the tests on a single thread, not taking advantage of possible paralellization.

image

aam avatar May 08 '24 19:05 aam