Integration with Paparazzi screenshot testing

[marcorighini]

Showkase is actually providing an integration with facebook screenshot testing. Is it possible to integrate Showkase with Paparazzi (screenshot testing without device, link https://github.com/cashapp/paparazzi)?

[vinaygaba]

Showkase itself doesn't provide any integrations (yet). The blog post that you probably saw(https://proandroiddev.com/automatic-screenshot-testing-for-all-your-compose-previews-6add202fecc7) used Showkase for the automated infra but then integrated it with Facebook's library. In theory, it can work with anything but you will have to do that setup today. I fully intend to provide a sample integration and expose it as an artifact at some point.

If you are interested, I'd be happy to accept a contribution that adds a Paparazzi integration submodule (or even Shot).

[davidvavra]

Wow, Paparazzi looks interesting. I'm not very happy with Facebook's library. It's not very maintained and it doesn't work for API 28+. If it runs without emulator, it can faster and a deal-breaker for CI integrations. Emulators on CI are pain to setup. I might try to play with Papparazi + Showkase in some free time.

[davidvavra]

I tried to play with Paparazzi in my sample app and when I run the Paparazzi test command, I get this:

java.lang.IllegalStateException: No instrumentation registered! Must run under a registering instrumentation.
	at androidx.test.platform.app.InstrumentationRegistry.getInstrumentation(InstrumentationRegistry.java:45)
	at androidx.test.runner.permission.PermissionRequester.<init>(PermissionRequester.java:67)
	at androidx.test.rule.GrantPermissionRule.<init>(GrantPermissionRule.java:69)
	at androidx.test.rule.GrantPermissionRule.grant(GrantPermissionRule.java:88)
	at me.vavra.screenshots.ComposePaparazziTests_ShowkaseCodegen.<init>(ComposePaparazziTests_ShowkaseCodegen.kt:23)

The root cause is that Showkase screenshot testing support requires to be run as instrumentation test. Why is that? Paparazzi provides context, layoutInflater etc. - maybe it can be used instead of instrumentation test requirement? In any way, Paparazzi support needs some changes in Showkase library.

The code I tried is in this branch: https://github.com/davidvavra/showkase-screenshot-tests/tree/paparazzi

[vinaygaba]

I think that's a Compose requirement itself. I haven't double checked this but that's what I always assumed.

[davidvavra]

Well, Papparazi has to do it somehow. They support snapshotting Views without instrumentation. They don't support Compose now, but they plan it for v1. But I'm assuming wrapping composables into Views might work. Might be worth investigating how they do it.

[mboudraa]

They embed the android studio renderer and rely on it to render views as images without requiring an emulator

[marcorighini]

I started the integration of paparazzi 1.0.0 (snapshot, the version supporting compose) in https://github.com/marcorighini/Showkase/tree/paparazzi-screenshot-testing-integration

Still to investigate:

• codegen not run on MyScreenshotTest file (unit test file)
• remove ComposeTestRule from current codegen and assess if needs to be replaced with Paparazzi rule

[alexvanyo]

I was able to utilize Showkase's code generation to automatically populate Paparazzi tests by avoiding the showkase-screenshot-testing module.

Instead, I just pulled out the list of components (from Showkase.getMetadata().componentList), and used that as the input to a parameterized Paparazzi test, using the componentKey for the test name and component as the @Composable under test.

[vinaygaba]

@alexvanyo Very cool! I had initially taken the parameterized test route myself but eventually found a limitation that didn't allow us to use it with our setup. It was something to do with how the Firebase Test Lab was handling sharding iirc.

[alexvanyo]

Just curious, were you trying to run those Paparzzi + Showkase tests on device? With Paparazzi, it can just be done off-device (with a parameterized test in src/test)

[vinaygaba]

@alexvanyo No I meant for the original implementation of automated screenshot testing itself. It wasn't using Paparazzi then but I think this might be viable now that Paparazzi supports Compose.

[alexvanyo]

Ah gotcha, yeah parameterized testing doesn't play well with instrumented tests last time I checked, I am using https://github.com/google/TestParameterInjector for that.

[davidvavra]

@alexvanyo Great! Can you share some code sample? I would like to play around with it and maybe then publish some article/library for it. I would love to get rid of Android emulator in our CI setup, it is causing a lot of flakiness/slowness/maintanance.

[vinaygaba]

Just as an update to an earlier comment on this thread, I did push a PR that adds Shot integration as a new artifact - https://github.com/airbnb/Showkase/pull/232. This should give you an end-to-end working solution for screenshot testing. Would love to have a similar artifact for Paparazzi as well...

[alexvanyo]

https://gist.github.com/alexvanyo/c4d2b6530f860f3c89a33324d4152ae8 has the main ScreenshotTest class with the integration.

[NapasPayu]

@alexvanyo I couldn't make it work. I'd be great if you can share all the related classes or a sample repo.

[alexvanyo]

https://github.com/android/nowinandroid/pull/101 has a full PR in a project with the setup with Paparazzi for reference!

[davidvavra]

Thanks @alexvanyo, I created a full-fledged example including CI integration based on your work:

https://github.com/davidvavra/showkase-screenshot-tests

And I wrote an article about it: https://proandroiddev.com/no-emulator-needed-for-screenshot-tests-of-compose-previews-on-ci-d292fa0eb26e

[vinaygaba]

A sample implementation has also been added to Showkase itself for your reference - https://github.com/airbnb/Showkase/blob/master/showkase-screenshot-testing-paparazzi-sample/src/test/java/com/airbnb/android/showkase/screenshot/testing/paparazzi/sample/PaparazziSampleScreenshotTest.kt

[vinaygaba]

I've just started working on an artifact (showkase-screenshot-testing-paparazzi) that will make it trivial to have a Showkase + Paparazzi setup for your codebase. It essentially auto generates the Paparazzi test class that leverages Showkase. All you need is 3 lines of code. PR here in case you have feedback - https://github.com/airbnb/Showkase/pull/294