node-core-utils icon indicating copy to clipboard operation
node-core-utils copied to clipboard

[WIP] feat: mark and skip flaky test

Open marco-ippolito opened this issue 1 year ago • 1 comments

Draft implementation to add flaky tests automatically to the .status file. Missing:

  • ~~rules to determine which tests we consider flaky~~
  • tests
  • proper matching for OS and architecture

I'm assuming that if the failed test shows up in at least 5 pr is flaky (debatable) and can be skipped, I'd increase this number to make sure no false positive

marco-ippolito avatar Oct 18 '23 16:10 marco-ippolito

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (6d68c99) 83.03% compared to head (1a12479) 82.95%. Report is 15 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #746      +/-   ##
==========================================
- Coverage   83.03%   82.95%   -0.08%     
==========================================
  Files          37       37              
  Lines        4178     4190      +12     
==========================================
+ Hits         3469     3476       +7     
- Misses        709      714       +5     

see 2 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Oct 20 '23 14:10 codecov[bot]