[Bug]: Ignore screenshot tests when running regular local tests

[JoseAlcerreca]

Is there an existing issue for this?

• [X] I have searched the existing issues

Is there a StackOverflow question about this issue?

• [X] I have searched StackOverflow

What happened?

When running the test tasks, screenshot tests are ignored: they simply pass but they do execute, which takes some time.

• Discuss: confirm that we want to keep screenshot tests out of the default test tasks. Roborazzi has a flag to include them by default (roborazzi.test.verify=true)
• Find the best way to ignore these tests, ideally so that no flags need to be added to the test execution commands on CI. Maybe @takahirom knows?
• Implement!

Relevant logcat output

No response

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[takahirom]

Thank you for reaching out and considering this issue.

If you want to ignore screenshot tests during local test runs, you can use the following configuration:

// Use -PincludeScreenshot to include screenshot tests
it.useJUnit {
    if (!project.hasProperty("includeScreenshot")) {
        excludeCategories("com.google.samples.apps.nowinandroid.testing.category.ScreenshotTests")
    }
}

By excluding the -PincludeScreenshot property, the screenshot tests will be ignored as needed. We have a similar setting in DroidKaigi app.

Roborazzi doesn't support this filtering in the library due to simplicity, using JUnit's built-in features. More information can be found at Roborazzi issue #36. It might be a bit confusing, but the flag roborazzi.test.verify=true enables you to call captureRoboImage() and generate an error if the images differ. This allows you to achieve the same behavior as ./gradlew verifyRoborazziDebug simply by running the ./gradlew testDebugUnitTest task. It's not meant for filtering.

I think the above solution should meet your requirements, but there might be a better way to approach this.

[JoseAlcerreca]

It might be a bit confusing, but the flag roborazzi.test.verify=true enables you to call captureRoboImage() and generate an error if the images differ. This allows you to achieve the same behavior as ./gradlew verifyRoborazziDebug simply by running the ./gradlew testDebugUnitTest task. It's not meant for filtering.

Yes, this is what I meant. I see that as the alternative to the solution proposed in this issue: instead of filtering, just run one task. However, I'd like to keep them separate.

I think the above solution should meet your requirements, but there might be a better way to approach this.

I'll look into this, thanks!