spring-aot-smoke-tests icon indicating copy to clipboard operation
spring-aot-smoke-tests copied to clipboard

Avoid GitHub sending build failure notifications for smoke tests that are not expected to pass

Open wilkinsona opened this issue 1 year ago • 0 comments

We have a number of smoke tests that are not expected to pass. Right now, each daily run of these smoke tests fails and causes GitHub to send a notification:

Notifications for scheduled workflows are sent to the user who last modified the cron syntax in the workflow file

These notifications are unwanted noise as they aren't really telling us anything. It would be useful if we could configure, on a per-test basis, that the expected outcome is failure. Something like this in an individual test's build.gradle:

aotSmokeTest {
    expectedOutcomes {
        nativeAppTest = failure
        nativeTest = failure
    }
}

This would translate to setting continue-on-error in the relevant step of smoke-test-jvm.yml and/or smoke-test-native.yml using an input from the calling workflow.

If we wish, we could then go a step further and fail the build if a test passes unexpectedly by adding a subsequent step that fails if the step that ran the test passed unexpectedly.

wilkinsona avatar Mar 27 '24 08:03 wilkinsona