flame icon indicating copy to clipboard operation
flame copied to clipboard

flame_test testGoldens needs to allow for a threshold

Open maks opened this issue 2 years ago • 3 comments

There are unfortunately multiple open issues in the Flutter SDK due to Golden tests failing with tiny differences due to running on different architectures and OS's:

  • https://github.com/flutter/flutter/issues/111739
  • https://github.com/flutter/flutter/issues/108399

A workaround is described in this article which allows for setting a threshold for Golden Tests. Because flmaee_test package defines its own testGolden() it would be useful to modify that to allow for setting a threshold to prevent these Goldentest visually insignificant failures due to machine arch or OS differences.

maks avatar May 02 '23 01:05 maks

Do you want to work on this? We only run and generate goldens on one platform to make sure they are exact, so we don't need this internally, but I see how it would be useful for others.

spydon avatar May 02 '23 06:05 spydon

Thanks for being open to this @spydon ! Yes I'm happy to give it a go. I actually ran into this with the Sashimi package, which is using testGolden() in its own tests.

maks avatar May 02 '23 06:05 maks

I assigned you to the issue :)

spydon avatar May 02 '23 06:05 spydon