opensearch-build
opensearch-build copied to clipboard
[Bug]: Flaky alerting dashboards cypress tests during Jenkins executions
Describe the bug
When the jenkins job runs the cypress tests in the Functional test repo, we found that 2 alerting dashboards cypress tests are flaky when executed using a security-enabled cluster. As described in this comment, we can see that each of the tests passes in previous test runs. https://github.com/opensearch-project/alerting-dashboards-plugin/issues/975#issuecomment-2174388518
To reproduce
We have difficulty reproducing the flakiness locally. The 2 tests that are called out as failing in the issue above pass reliably when executed locally using a security-enabled cluster.
- Use docker to create a security-enabled domain with the frontend running
- Use the steps in the functional test repo developer guide to execute the tests against the docker cluster - https://github.com/opensearch-project/opensearch-dashboards-functional-test/blob/main/DEVELOPER_GUIDE.md
Expected behavior
No response
Screenshots
This test fail in this instance because the "select your tenant" window wasn't closed.
This test failed because it timed out waiting for the UI to load. https://ci.opensearch.org/ci/dbc/integ-test-opensearch-dashboards/2.15.0/7742/linux/x64/deb/test-results/5966/integ-test/alertingDashboards/with-security/cypress-screenshots/plugins/alerting-dashboards-plugin/document_level_monitor_spec.js/DocumentLevelMonitor+--+can+be+created+--+by+extraction+query+editor+--+before+each+hook+%28failed%29.png
This test failed because it timed out waiting for the UI to load.
Host / Environment
No response
Additional context
No response
Relevant log output
No response