wpt.fyi icon indicating copy to clipboard operation
wpt.fyi copied to clipboard

Warn about bad tests vs bad implementations

Open fantasai opened this issue 5 years ago • 0 comments

It might be useful to remind people (especially new people) that tests can be failing due to bad tests as well as bad implementations, so my suggestion is to

  • Move the “Showing 38733 tests (1733336 subtests) from the latest master test runs for chrome[experimental], edge[experimental], firefox[experimental], safari[experimental]” to after the results table (and/or come up with UI for it that isn't so attention-grabbing and doesn't repeat information already in the table headers).

  • Add in its place a disclaimer like “Failing tests can be caused by a bad implementation or by a bad test. Please check the relevant specifications to determine which.” so that less-experienced people landing on wpt.fyi realize that tests can also be buggy and failures are not necessarily those of the implementations.

fantasai avatar Oct 30 '20 18:10 fantasai