OpenSearch-Dashboards icon indicating copy to clipboard operation
OpenSearch-Dashboards copied to clipboard

[ACTION NEEDED] Fix flaky integration tests at distribution level

Open gaiksaya opened this issue 1 year ago • 5 comments

What is the bug? It was observed in 2.13.0 and previous other releases that this component manually signed off on the release for failing integration tests. See https://github.com/opensearch-project/opensearch-build/issues/4433#issuecomment-2026486279 The flakiness of the test runs take a lot of time from the release team to collect go/no-go decision and significantly lower the confidence in the release bundles.

How can one reproduce the bug? Steps to reproduce the behavior:

  1. Run integration testing for altering and see the failures.
  2. Issues can be reproduced using the steps declared in AUTOCUT issues for failed integration testing

What is the expected behavior? Tests should be consistently passing.

Do you have any additional context? Please note that this is a hard blocker for 2.14.0 release as per the discussion here

gaiksaya avatar Apr 03 '24 21:04 gaiksaya

Flakey test from 2.13: https://github.com/opensearch-project/OpenSearch-Dashboards/issues/6373, https://github.com/opensearch-project/opensearch-dashboards-functional-test/pull/1210

ashwin-pc avatar Apr 09 '24 01:04 ashwin-pc

@ashwin-pc @AMoo-Miki Can you please provide your inputs?

bbarani avatar Apr 23 '24 22:04 bbarani

Both the tests should already be resolved. Will let @kavilla reply once he verifies it

ashwin-pc avatar Apr 23 '24 22:04 ashwin-pc

Adding 2.14.0 release manager @rishabh6788

gaiksaya avatar Apr 30 '24 20:04 gaiksaya

Checked previously and passing will make check when RC is ran.

kavilla avatar May 02 '24 07:05 kavilla