[AUTOCUT] Gradle Check Flaky Test Report for ClientYamlTestSuiteIT

[opensearch-ci-bot]

Flaky Test Report for ClientYamlTestSuiteIT

Noticed the ClientYamlTestSuiteIT has some flaky, failing tests that failed during post-merge actions.

Details

Git Reference	Merged Pull Request	Build Details	Test Name
0df6e9ed7e35646dbce604a6c88c4ace706b797f	15098	43940	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=indices.delete_index_template/10_basic/Delete index template which is not used by data stream but index pattern matches}
c85ee6898b4e3e2f4a2037e42337a7eb8033de9e	15988	48303	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/360_date_histogram/Date histogram aggregation w/ shared field range test}
481fd38c807024bfdcbc28a5b3ddb67fbfc439ef	16098	48529	org.opensearch.test.rest.ClientYamlTestSuiteIT.classMethod org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=indices.segments/10_basic/basic segments test}
d64c5f800b674427cd18c31d225d7312f82cd57a	15866	47801	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/360_date_histogram/Date histogram aggregation w/ shared field range test}
2742e745baa3db474b7a10a66b3ebf461a21fdad	15700	47023	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/360_date_histogram/Date histogram aggregation w/ shared field range test}
7a54f7193005adeb6c09f5a471db19f31bd51008	14707	42236	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
8b629484277b161e01fd6affc8ab7ab2f580bbeb	13953	39797	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
ed24a14f7dba4adbc14639c2b0e6f2bcff81be1e	14881	43113	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
fdd3ea769e3f992bbf85b0236b8c91ad168a7df0	15023	43647	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}

The other pull requests, besides those involved in post-merge actions, that contain failing tests with the ClientYamlTestSuiteIT class are:

• 15024
• 14443
• 15124
• 15568
• 16294
• 15131
• 13853
• 13829
• 15333
• 15258
• 15922
• 13865
• 13788
• 13821
• 13620
• 14446
• 13232
• 13461
• 14383
• 15039
• 15012
• 15365
• 14069
• 14464
• 15545
• 14454
• 15289
• 14718
• 14774
• 15882
• 14613
• 14711
• 13573
• 14486
• 15194
• 15410
• 15509
• 15607
• 14817
• 15145
• 15256
• 15557
• 15603
• 13190
• 13929
• 14611
• 14630
• 15100
• 15153
• 15358
• 15381
• 15578
• 16038
• 14513
• 14415
• 12782
• 13772
• 13888
• 14118
• 14195
• 14684
• 15128
• 15326
• 15430
• 15649
• 15679

For more details on the failed tests refer to OpenSearch Gradle Check Metrics dashboard.

[getsaurabh02]

@prudhvigodithi can we confirm if these failures are pointing to an individual test or the overall test class?

[getsaurabh02]

Also, can we highlight the branch the test is failing on, in the issue description?

[prudhvigodithi]

Hey @getsaurabh02 today the gradle check runs on a commit ref, the issue description, the markdown table created has the commit that has the flaky failure. Here for this issue the commit https://github.com/opensearch-project/OpenSearch/commit/8b629484277b161e01fd6affc8ab7ab2f580bbeb has a flaky failure for the test class ClientYamlTestSuiteIT and the failing test is org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}.

Sometimes there are can be more than one test failing for the same commit or for the different one, here is the example https://github.com/opensearch-project/OpenSearch/issues/14381, in that case we group them under one issue created for a specific test class name.

[sandeshkr419]

@finnegancarroll do you want to take a look on these failures?

[sandeshkr419]

@prudhvigodithi Does the ci-bot closes out flaky tests if they don't fail for a certain time / runs?

[prudhvigodithi]

Hey @sandeshkr419 the automation wont re-open the issue once closed. So the idea is once the flaky tests are fixed, the user can close the issue and automation wont re-open if there are no new failures with respect to same flaky test. For now we can close the issue and monitor this flaky test issue. Thanks