flutter icon indicating copy to clipboard operation
flutter copied to clipboard

Linux module_test is 2.06% flaky

Open fluttergithubbot opened this issue 1 year ago • 6 comments

The post-submit test builder Linux module_test had a flaky ratio 2.06% for the past (up to) 100 commits, which is above our 2.00% threshold.

One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20module_test/17882 Commit: https://github.com/flutter/flutter/commit/609d30fe5859db3a05c550174fcf7709a33f4888

Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20module_test/17882 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20module_test/17837

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20module_test

Please follow https://github.com/flutter/flutter/wiki/Reducing-Test-Flakiness#fixing-flaky-tests to fix the flakiness and enable the test back after validating the fix (internal dashboard to validate: go/flutter_test_flakiness).

fluttergithubbot avatar Feb 28 '24 10:02 fluttergithubbot

The failure for https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20module_test/17882 is:

[2024-02-27 11:27:35.856598] [STDOUT] stdout: Warning: An error occurred while preparing SDK package Android SDK Platform 29: Archive is not a ZIP archive.:
[2024-02-27 11:27:35.856746] [STDOUT] stdout: java.util.zip.ZipException: Archive is not a ZIP archive
[2024-02-27 11:27:35.856792] [STDOUT] stdout: 	at org.apache.commons.compress.archivers.zip.ZipFile.positionAtEndOfCentralDirectoryRecord(ZipFile.java:1141)
[2024-02-27 11:27:35.856838] [STDOUT] stdout: 	at org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory(ZipFile.java:1021)
[2024-02-27 11:27:35.856885] [STDOUT] stdout: 	at org.apache.commons.compress.archivers.zip.ZipFile.populateFromCentralDirectory(ZipFile.java:702)
[2024-02-27 11:27:35.856921] [STDOUT] stdout: 	at org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java:371)
[2024-02-27 11:27:35.856961] [STDOUT] stdout: 	at org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java:318)
[2024-02-27 11:27:35.856994] [STDOUT] stdout: 	at org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java:274)

And for the other one:

[2024-02-23 16:16:02.794927] [STDOUT] stderr: Exception in thread "main" java.net.ConnectException: Connection timed out
[2024-02-23 16:16:02.795434] [STDOUT] stderr: 	at java.base/sun.nio.ch.Net.connect0(Native Method)
[2024-02-23 16:16:02.795543] [STDOUT] stderr: 	at java.base/sun.nio.ch.Net.connect(Net.java:579)
[2024-02-23 16:16:02.795566] [STDOUT] stderr: 	at java.base/sun.nio.ch.Net.connect(Net.java:568)

These both look network related to me. We should catch both these error messages in the tool gradle_errors handling.

christopherfujino avatar Mar 05 '24 21:03 christopherfujino

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-03-05 and 2024-03-12 is 0.00%. Flaky number: 0; total number: 99.

fluttergithubbot avatar Mar 13 '24 09:03 fluttergithubbot

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-03-13 and 2024-03-19 is 0.00%. Flaky number: 0; total number: 99.

fluttergithubbot avatar Mar 20 '24 09:03 fluttergithubbot

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-03-19 and 2024-03-25 is 0.00%. Flaky number: 0; total number: 98.

fluttergithubbot avatar Mar 27 '24 09:03 fluttergithubbot

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-03-27 and 2024-04-02 is 0.00%. Flaky number: 0; total number: 99.

fluttergithubbot avatar Apr 03 '24 09:04 fluttergithubbot

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-04-03 and 2024-04-09 is 0.00%. Flaky number: 0; total number: 95.

fluttergithubbot avatar Apr 10 '24 09:04 fluttergithubbot

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-04-08 and 2024-04-16 is 0.00%. Flaky number: 0; total number: 98.

fluttergithubbot avatar Apr 17 '24 09:04 fluttergithubbot

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-04-18 and 2024-04-23 is 1.03%. Flaky number: 1; total number: 97. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20module_test/18797 Commit: https://github.com/flutter/flutter/commit/4c46030927626782555918be39985ff95de83438 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20module_test/18797

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20module_test

fluttergithubbot avatar Apr 24 '24 09:04 fluttergithubbot

@christopherfujino There is only one flake of this test in the past >700 runs. Should we close this issue?

zanderso avatar Apr 25 '24 15:04 zanderso

@christopherfujino There is only one flake of this test in the past >700 runs. Should we close this issue?

SGTM

christopherfujino avatar Apr 26 '24 21:04 christopherfujino

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

github-actions[bot] avatar May 10 '24 22:05 github-actions[bot]