wpt-metadata icon indicating copy to clipboard operation
wpt-metadata copied to clipboard

Populate flaky tests from recent known identical runs

Open foolip opened this issue 6 years ago • 2 comments

There was just a series of 10 commits in wpt that didn't change any actual tests. All of them triggered full Chrome and Firefox runs, and from those we can see which tests had different outcomes:

This is similar to what we can see from https://wpt.fyi/insights, but we can have even higher confidence in these differences.

@KyleJu can you populate this repo with the results from above, to get a very good start on flakes that occur often enough to show up in 10 runs?

foolip avatar Oct 10 '19 09:10 foolip

@KyleJu does https://github.com/web-platform-tests/wpt-metadata already support the format for flaky metadata? I search for flak in the repo but found nothing.

foolip avatar Oct 21 '19 07:10 foolip

@foolip Not yet the implementation will start later this quarter

KyleJu avatar Oct 28 '19 15:10 KyleJu