wpt.fyi icon indicating copy to clipboard operation
wpt.fyi copied to clipboard

Filtering can lead to misleading "missing" tests

Open foolip opened this issue 2 years ago • 0 comments

@nairnandu pointed out that this search has missing results for Chrome: https://wpt.fyi/results/pointerevents?q=label%3Ainterop-2023-events%20%28chrome%3A%21ok%26chrome%3A%21pass%29%20none%28triaged%3Achrome%29&run_id=5196887768956928&run_id=5117608511143936&run_id=6246543416360960

image

I'm not sure why it looks like this, but removing (chrome:!ok&chrome:!pass) none(triaged:chrome) reveals that there are no missing tests: https://wpt.fyi/results/pointerevents?q=label%3Ainterop-2023-events&run_id=5196887768956928&run_id=5117608511143936&run_id=6246543416360960

I think this is most likely due to the search cache not returning any results for Chrome, but some results for Firefox and Safari, which are then merged. Normally (when not filtering) it's correct to say that test results are missing, but in this case it's misleading.

@KyleJu WDYT?

foolip avatar Jun 13 '23 07:06 foolip