Linux_android flutter_engine_group_performance is 3.30% flaky

[fluttergithubbot]

The post-submit test builder Linux_android flutter_engine_group_performance had a flaky ratio 3.30% for the past (up to) 100 commits, which is above our 2.00% threshold.

One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android%20flutter_engine_group_performance/8896 Commit: https://github.com/flutter/flutter/commit/fae99c5a7ca85b54534f34fce7830bd05d4b5c22

Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android%20flutter_engine_group_performance/8896 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android%20flutter_engine_group_performance/8891 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android%20flutter_engine_group_performance/8887

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_android%20flutter_engine_group_performance

Please follow https://github.com/flutter/flutter/wiki/Reducing-Test-Flakiness#fixing-flaky-tests to fix the flakiness and enable the test back after validating the fix (internal dashboard to validate: go/flutter_test_flakiness).

[zanderso]

Looking at the logs, I'm not able to discern why the tests failed. Routing to the Android team.

[camsim99]

The failure seems to be

Failure [INSTALL_FAILED_UPDATE_INCOMPATIBLE: Package dev.flutter.multipleflutters signatures do not match the previously installed version; ignoring!]

Looking this up, it seems that this may be caused by failure to uninstall an older version of the application or its data (example Stack Overflow answer). @godofredoc could this be an infra issue?

[godofredoc]

Is the setup of the test removing the old version of the application? or is there a crash when the test is trying to remove the application?

[camsim99]

It looks like the test is installing the app, then runs some test, and then uninstalls the app. It crashes trying to install the app. If the failure means what I thought, then maybe it's an issue with the app not properly being uninstalled after running? That happens here.

[godofredoc]

Should we try to uninstall the app before installing it?

Here is a potential scenario:

• Another test that is not doing proper cleanup is leaving the application installed, then this test is building the application with different keys and failing to install it because of a signature mismatch.

This may be a consequence of the fix for https://github.com/flutter/flutter/issues/126557 which will force new keys even if running on the same bot.

[camsim99]

Ah, I see. Yeah, let's try it! I'll make this change then update here.

Edit: https://github.com/flutter/flutter/pull/127429

[godofredoc]

@keyonghan can we also add the dev.flutter.multipleflutters namespace to the low level device cleanup in device_doctor?

[keyonghan]

@keyonghan can we also add the dev.flutter.multipleflutters namespace to the low level device cleanup in device_doctor?

The apps uninstall logic is only implemented in ios devices. we can do similar things for android. Created https://github.com/flutter/flutter/issues/127237 to track.

[fluttergithubbot]

[prod pool] current flaky ratio for the past (up to) 100 commits is 3.09%. Flaky number: 3; total number: 97. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android%20flutter_engine_group_performance/9028 Commit: https://github.com/flutter/flutter/commit/f6f5bb9023af3569626c6c6d7b2deb646bd716dd Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android%20flutter_engine_group_performance/9028 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android%20flutter_engine_group_performance/8978 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android%20flutter_engine_group_performance/8966

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_android%20flutter_engine_group_performance

[camsim99]

An attempt to fix this is waiting on issues in the tree to land: https://github.com/flutter/flutter/pull/127429.

[fluttergithubbot]

[prod pool] current flaky ratio for the past (up to) 100 commits is 5.15%. Flaky number: 5; total number: 97. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android%20flutter_engine_group_performance/9175 Commit: https://github.com/flutter/flutter/commit/59128bd61e06ca2a30392640184c13060f1e7775 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android%20flutter_engine_group_performance/9175 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android%20flutter_engine_group_performance/9139 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android%20flutter_engine_group_performance/9138 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android%20flutter_engine_group_performance/9125 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android%20flutter_engine_group_performance/9098 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_android%20flutter_engine_group_performance/9082

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_android%20flutter_engine_group_performance

[Hixie]

There has not been another update from @fluttergithubbot since https://github.com/flutter/flutter/pull/127429 landed.

[fluttergithubbot]

[prod pool] current flaky ratio for the past (up to) 100 commits is 0.00%. Flaky number: 0; total number: 96.

[camsim99]

Closing for now as https://github.com/flutter/flutter/pull/127429 seems to have been a valid fix.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.