results-analysis icon indicating copy to clipboard operation
results-analysis copied to clipboard

Published 20 hours ago • verified publisher icon web-platform-tests

Fix #88: Mark as bad any run with >=66 missing results

Open gsnedders opened this issue 3 years ago • 3 comments

This is a high enough threshold that it doesn't exclude runs with missing results (e.g., due to -print or -crash being unsupported), but does catch the vast majority with an unusually high number of missing results.

gsnedders avatar Jul 07 '22 16:07 gsnedders

https://gist.github.com/gsnedders/676cd70e21bcd8d96a79550582d455b3 for a diff of the output.

gsnedders avatar Jul 07 '22 18:07 gsnedders

Oh, actually, maybe my data is bad here; seems to omit cases where entire trees are missing.

gsnedders avatar Jul 07 '22 19:07 gsnedders

@gsnedders what are some of the runs that you're hoping this will exclude? Any idea what went wrong with them?

I would quite like it if we used the manifest when processing results in wpt.fyi, so that we could tell if a run is complete or not and label incomplete runs, or at least know for sure how many tests of which type are missing.

foolip avatar Jul 08 '22 07:07 foolip