devtools icon indicating copy to clipboard operation
devtools copied to clipboard

Some tests fail locally on macOS

Open DanTup opened this issue 5 years ago • 1 comments

While trying to fix up test failures on another branch, I noticed that some of them failed even without my changes and when running on Flutter dev:

Screenshot 2020-10-06 at 10 59 34

I don't know if these only occur locally, only for me, or generally on macOS (macOS bots are currently disabled), but it'd be good to get them working to simplify running tests locally (and knowing whether you've broken anything).

1. inspector_controller_test

This fails because an isolate event arrives and is sent to the message bus, but the message bus is null.

Q: Should the test set up a message bus, or should _sendToMessageBus handle a null bus?

2. dart_cli_profile_test

Exception: There are no registered methods for service "_flutter.listViews

This is not running on CI because it's in an integration folder, but the rest are in integration_tests.

Q: Should this be in integration_tests? Q: Why isn't the service registered? 🤷‍♂️

3. memory_screen_test

Exception: Getting current working directory failed, path = '' (OS Error: No such file or directory, errno = 2)

Unclear why this _fs.currentDirectory is throwing this error for me, but a try/catch around the code and falling back to the temp directory solves it.

Q: Is it reasonable to commit the try/catch?

4. timeline_flame_chart_test

Golden failure (no obvious difference to my eyes - perhaps slight differences between platforms and these were generated on Linux?).

Q: Is this expected to pass on macOS? If not, should we skip when platform != Linux?

5. treemap_test

Expected: exactly one matching node in the widget tree Actual: _TextFinder:<zero widgets with text "new CastIterable. [0.2 KB]" (ignoring offstage widgets)> Which: means none were found but one was expected

This fails on:

expect(find.text('new CastIterable. [0.2 KB]'), findsOneWidget);

Though if I run the test on a device and add a breakpoint, I can see that text in the last item of the breadcrumb, so it's not clear why this isn't found.

DanTup avatar Oct 06 '20 10:10 DanTup

From a discussion with Danny, some of these failures might be race conditions, and some are tests that we're not running on the bots (but probably should).

devoncarew avatar Oct 08 '20 15:10 devoncarew

We have not seen this issue in a while - closing as obsolete

kenzieschmoll avatar Jun 26 '23 16:06 kenzieschmoll