About RoborazziTaskType parameter to RoborazziOptions

[takahirom]

Motivation

I created Roborazzi because I wanted to view layouts during UI tests. This was necessary as Robolectric did not provide layout visibility. Currently, Roborazzi doesn't support merely viewing layouts during the verification task (roborazziVerifyDebug). I also believe that these debugging screenshots shouldn't cause a failure in UI tests, such as checking button visibility, as the screenshots offer supplementary information.

Proposal

To modify the behavior of captureRoboImage(), I'm considering the addition of a taskType parameter to RoborazziOptions.

      onView(ViewMatchers.isRoot())
        .captureRoboImage(
          roborazziOptions = RoborazziOptions(
            taskType = RoborazziTaskType.Verify
          )
        )

This change enables image comparison during the verification task.

      onView(ViewMatchers.isRoot())
        .captureRoboImage(
          roborazziOptions = RoborazziOptions(
            taskType = roborazziSystemPropertyTaskType().convertVerifyingToComparing()
          )
        )

Tradeoffs

There is a risk that users might misuse Roborazzi. If used without the Roborazzi plugin, screenshots will be created, but without the plugin, no test report can be generated.

The Roborazzi plugin needs to accommodate various types of screenshot tests, even when a specific task is being used.

Pull request

https://github.com/takahirom/roborazzi/pull/211

[shashachu]

This task type doesn't seem to be getting set inside of Android Studio for me, via gradle.properties. I have roborazzi.test.record=true in my gradle.properties but task type seems to always be set to None (verified via debugging). Is there something I'm doing wrong?

[takahirom]

@shashachu Are you applying the Roborazzi Gradle plugin? The Roborazzi plugin passes project parameters to the test. You can also check the Roborazzi logs of the Gradle plugin passing parameters by running with --info.

[shashachu]

I can double check on Monday but I believe so? Wouldn't I need the plugin to run roborazzi at all?

[takahirom]

You can run Roborazzi without the plugin, but you need to add system properties to tests manually. Additionally, you won't be able to see the HTML report, and it could potentially break the Gradle cache.

[shashachu]

We are using the roborazzi plugin in the modules that have snapshot tests. does it need to be applied at the top level?

[takahirom]

It doesn't need to apply at the top level. Could you check what Roborazzi passes to the test with the --info parameter, like ./gradlew testDebugUnitTest --info?

[shashachu]

It doesn't need to apply at the top level. Could you check what Roborazzi passes to the test with the --info parameter, like ./gradlew testDebugUnitTest --info?

Thanks @takahirom. This was user error; the application of the gradle plugin was behind a flag (we do this to save on configuration time with a lot of our non-essential plugins) so the parameters were not being passed through. Perhaps the docs can be clarified to state that running under AS requires using the Gradle plugin.

[takahirom]

I personally haven't done that configuration. But we could show a warning or error when we can't find any plugins that we can use.