OpenSearch icon indicating copy to clipboard operation
OpenSearch copied to clipboard

[Gradle Check Retry] Publish/Record tests which pass after a retry to address flakiness

Open kotwanikunal opened this issue 3 years ago • 2 comments

Is your feature request related to a problem? Please describe.

  • gradle check test failures are now retried as a part of https://github.com/opensearch-project/OpenSearch/pull/2638
  • This issue addresses the scenarios where the check passes with retries where the flakey tests are lost under the green status

Describe the solution you'd like

  • As a part of the solution, there should be a record of logs and/or a comment on the PR in case of test failures
  • The generated report has a log of the tests that fail even though the overall job succeeds
  • Record the frequent offenders to address and mitigate the underlying root cause for flakiness

Describe alternatives you've considered

  • Manually keeping track of failed tests from the gradle check bot reports

Additional context

  • N/A

kotwanikunal avatar Apr 06 '22 21:04 kotwanikunal

Hi @kotwanikunal,

I'd like to contribute to this issue!

yeohbraddy avatar Sep 15 '22 13:09 yeohbraddy

@yeohbraddy Thanks. Let me know if you have any queries around this.

kotwanikunal avatar Sep 20 '22 23:09 kotwanikunal