wpt.fyi icon indicating copy to clipboard operation
wpt.fyi copied to clipboard

Reftest analyser sometimes shows zero diff of failing test

Open gsnedders opened this issue 2 years ago • 1 comments

Following the link from https://wpt.fyi/results/css/CSS2/borders/border-left-color-132.xht?run_id=5142037970550784&run_id=5093854074634240 results in:

maxDifference: 255 totalPixels: 479972

… which explains the fail. But only sometimes. Otherwise it shows no difference. No idea what's up with this.

gsnedders avatar Apr 26 '23 14:04 gsnedders

FWIW, this shows what I'm seeing:

Screenshot 2023-04-26 at 18 10 04

I'm starting to wonder if it believes almost every pixel on the actual screenshot is RGB(0, 0, 0)… which also seems wrong.

And I'm getting different results in every browser. Firefox Nightly says:

maxDifference: 0 totalPixels: 0

Chrome Canary says:

maxDifference: 1 totalPixels: 18432

And STP says:

maxDifference: 255 totalPixels: 479972

gsnedders avatar Apr 26 '23 17:04 gsnedders