opensearch-build icon indicating copy to clipboard operation
opensearch-build copied to clipboard

[Bug]: Flaky alerting dashboards cypress tests during Jenkins executions

Open AWSHurneyt opened this issue 8 months ago • 2 comments

Describe the bug

When the jenkins job runs the cypress tests in the Functional test repo, we found that 2 alerting dashboards cypress tests are flaky when executed using a security-enabled cluster. As described in this comment, we can see that each of the tests passes in previous test runs. https://github.com/opensearch-project/alerting-dashboards-plugin/issues/975#issuecomment-2174388518

To reproduce

We have difficulty reproducing the flakiness locally. The 2 tests that are called out as failing in the issue above pass reliably when executed locally using a security-enabled cluster.

  1. Use docker to create a security-enabled domain with the frontend running
  2. Use the steps in the functional test repo developer guide to execute the tests against the docker cluster - https://github.com/opensearch-project/opensearch-dashboards-functional-test/blob/main/DEVELOPER_GUIDE.md

Expected behavior

No response

Screenshots

This test fail in this instance because the "select your tenant" window wasn't closed. Screenshot 2024-06-18 at 1 17 41 PM

This test failed because it timed out waiting for the UI to load. https://ci.opensearch.org/ci/dbc/integ-test-opensearch-dashboards/2.15.0/7742/linux/x64/deb/test-results/5966/integ-test/alertingDashboards/with-security/cypress-screenshots/plugins/alerting-dashboards-plugin/document_level_monitor_spec.js/DocumentLevelMonitor+--+can+be+created+--+by+extraction+query+editor+--+before+each+hook+%28failed%29.png

This test failed because it timed out waiting for the UI to load. Screenshot 2024-06-18 at 4 29 53 PM

Host / Environment

No response

Additional context

No response

Relevant log output

No response

AWSHurneyt avatar Jun 18 '24 23:06 AWSHurneyt