flutter icon indicating copy to clipboard operation
flutter copied to clipboard

Mac tool_integration_tests_5_5 is 2.00% flaky

Open fluttergithubbot opened this issue 1 year ago • 1 comments

The post-submit test builder Mac tool_integration_tests_5_5 had a flaky ratio 2.00% for the past (up to) 100 commits, which is above our 2.00% threshold.

One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1049 Commit: https://github.com/flutter/flutter/commit/4f2211b550daef2d6073bb36542040f1f7156d51

Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1049 https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1022

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac%20tool_integration_tests_5_5

Please follow https://github.com/flutter/flutter/blob/master/docs/infra/Reducing-Test-Flakiness.md#fixing-flaky-tests to fix the flakiness and enable the test back after validating the fix (internal dashboard to validate: go/flutter_test_flakiness).

fluttergithubbot avatar Oct 16 '24 10:10 fluttergithubbot

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-10-23 and 2024-10-29 is 2.00%. Flaky number: 2; total number: 100. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1239 Commit: https://github.com/flutter/flutter/commit/5879a5cba13741a170c18a91f9583f6483c8dd60 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1239 https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1200

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac%20tool_integration_tests_5_5

fluttergithubbot avatar Oct 30 '24 09:10 fluttergithubbot

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-10-28 and 2024-11-05 is 1.00%. Flaky number: 1; total number: 100. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1325,%201324 Commit: https://github.com/flutter/flutter/commit/9a5e8ce6b52c8e8a8864270c45ea6a82497e7bb2 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1325 https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1324

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac%20tool_integration_tests_5_5

fluttergithubbot avatar Nov 06 '24 09:11 fluttergithubbot

We're seeing similar failures on this PR: https://github.com/flutter/flutter/pull/157100

bkonyi avatar Nov 06 '24 21:11 bkonyi

This looks related to https://github.com/dart-lang/native/issues/1698.

bkonyi avatar Nov 06 '24 22:11 bkonyi

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-11-04 and 2024-11-12 is 1.06%. Flaky number: 1; total number: 94. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1390 Commit: https://github.com/flutter/flutter/commit/2e23f7b30ea79180a3507c53058a21c2d856e4e5 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1390

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac%20tool_integration_tests_5_5

fluttergithubbot avatar Nov 13 '24 09:11 fluttergithubbot

(Triage): @bkonyi are we still tracking this? Thanks!

Piinks avatar Nov 19 '24 23:11 Piinks

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-11-12 and 2024-11-19 is 2.06%. Flaky number: 2; total number: 97. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1557 Commit: https://github.com/flutter/flutter/commit/966aeb28ea04d5166ad40a85ca20f7b093d70260 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1557 https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1520

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac%20tool_integration_tests_5_5

fluttergithubbot avatar Nov 20 '24 09:11 fluttergithubbot

I thought this was resolved as of https://github.com/flutter/flutter/pull/158946, but apparently we're still missing something... :(

bkonyi avatar Nov 20 '24 15:11 bkonyi

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-11-18 and 2024-11-26 is 2.00%. Flaky number: 2; total number: 100. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1625 Commit: https://github.com/flutter/flutter/commit/773b42f4fb80583db8f7424f3f787dd802ddfc21 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1625 https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1557

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac%20tool_integration_tests_5_5

fluttergithubbot avatar Nov 27 '24 09:11 fluttergithubbot

No failures since https://github.com/flutter/flutter/pull/159275 landed with additional logging, so no more information to go off of.

bkonyi avatar Nov 27 '24 15:11 bkonyi

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-11-21 and 2024-12-03 is 1.00%. Flaky number: 1; total number: 100. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1625 Commit: https://github.com/flutter/flutter/commit/773b42f4fb80583db8f7424f3f787dd802ddfc21 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1625

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac%20tool_integration_tests_5_5

fluttergithubbot avatar Dec 04 '24 09:12 fluttergithubbot

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-11-28 and 2024-12-10 is 0.00%. Flaky number: 0; total number: 99.

fluttergithubbot avatar Dec 11 '24 09:12 fluttergithubbot

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

github-actions[bot] avatar Dec 25 '24 21:12 github-actions[bot]