paparazzi icon indicating copy to clipboard operation
paparazzi copied to clipboard

Rendering differences between platforms seems to have increased with 1.3.4

Open VirtualParticle opened this issue 8 months ago • 24 comments

Description In previous versions, there have been slight variations between the Paparazzi screenshots generated on Linux CI runs and local MacOS runs. This required setting a maximum percent difference above zero, although I was able to set it very low (0.00005%) and still have it work. The primary difference seems to be around text anti-aliasing and color blending more generally.

In the latest version, however, it seems like the difference has increased. It might be something with the new version of layoutlib. Seems like this is happening on the Paparazzi repo itself too: https://github.com/cashapp/paparazzi/pull/1314/files#r1544924645

Steps to Reproduce

Here's an example of a test with a screenshot originally generated on MacOS and failing on Linux with Paparazzi 1.3.4 (this has a difference of 0.007134%):

@Test
fun example() {
    paparazzi.snapshot {
        Box(
            modifier = Modifier.background(Color(0xFF000033))
        ) {
            Text("ExampleText", color = Color.White)
        }
    }
}

image

With Paparazzi 1.3.3, the same test passes on Linux CI with the screenshot generated on MacOS, even with the maxPercentDifference set to 0.

When the accessibility overlay is enabled, the error increases to 0.083399%.

image

Expected behavior I'd expect the tests to be identical on all platforms but that wasn't the old behavior either. Acceptable behavior would just be to have the discrepancy be small enough to ignore. To deal with the higher difference scenarios, we'd have to set the maxPercentDifference to be high enough where actual bugs may be missed.

Additional information:

  • Paparazzi Version: 1.3.4
  • OS: MacOS (local) and Linux (CI)
  • Compile SDK: 34
  • Gradle Version: 8.7
  • Android Gradle Plugin Version: 8.4.1

VirtualParticle avatar May 30 '24 16:05 VirtualParticle