OpenSearch
OpenSearch copied to clipboard
opensearch-project
[BUG] UpgradeClusterClientYamlTestSuiteIT (p0=upgraded_cluster/10_basic/Find a task result record from the old cluster)
Describe the bug
Autocut failure from https://build.ci.opensearch.org/job/gradle-check/14632.
UpgradeClusterClientYamlTestSuiteIT (p0=upgraded_cluster/10_basic/Find a task result record from the old cluster) flaky.
Re-running succeeded.
Additional context
Suite: Test class org.opensearch.upgrades.UpgradeClusterClientYamlTestSuiteIT
1> [2023-04-28T18:42:18,799][INFO ][o.o.u.UpgradeClusterClientYamlTestSuiteIT] [test] [p0=upgraded_cluster/10_basic/Find a task result record from the old cluster] before test
1> [2023-04-28T18:42:18,863][INFO ][o.o.u.UpgradeClusterClientYamlTestSuiteIT] [test] initializing REST clients against [http://[::1/]:39129, http://127.0.0.1:46671/, http://[::1]:34885/, http://127.0.0.1:42455/, http://[::1]:39415/, http://127.0.0.1:37877/]
2> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
2> SLF4J: Defaulting to no-operation (NOP) logger implementation
2> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
1> [2023-04-28T18:42:19,367][INFO ][o.o.u.UpgradeClusterClientYamlTestSuiteIT] [test] initializing client, minimum OpenSearch version [3.0.0], cluster-manager version, [3.0.0], hosts [http://[::1/]:39129, http://127.0.0.1:46671/, http://[::1]:34885/, http://127.0.0.1:42455/, http://[::1]:39415/, http://127.0.0.1:37877/]
2> ะบะฒัั. 28, 2023 6:42:19 PM org.opensearch.client.RestClient logResponse
2> WARNING: request [GET http://[::1]:39129/.tasks/_search?rest_total_hits_as_int=true&error_trace=true] returned 1 warnings: [299 OpenSearch-3.0.0-SNAPSHOT-84227681555b4aec12725c4454c39834cb6839d2 "this request accesses system indices: [.tasks], but in a future major version, direct access to system indices will be prevented by default"]
1> [2023-04-28T18:42:19,548][INFO ][o.o.u.UpgradeClusterClientYamlTestSuiteIT] [test] Stash dump on test failure [{
1> "stash" : {
1> "body" : {
1> "took" : 100,
1> "timed_out" : false,
1> "_shards" : {
1> "total" : 1,
1> "successful" : 1,
1> "skipped" : 0,
1> "failed" : 0
1> },
1> "hits" : {
1> "total" : 0,
1> "max_score" : null,
1> "hits" : [ ]
1> }
1> }
1> }
1> }]
1> [2023-04-28T18:42:20,208][INFO ][o.o.u.UpgradeClusterClientYamlTestSuiteIT] [test] [p0=upgraded_cluster/10_basic/Find a task result record from the old cluster] after test
2> REPRODUCE WITH: ./gradlew ':qa:rolling-upgrade:v2.8.0#upgradedClusterTest' --tests "org.opensearch.upgrades.UpgradeClusterClientYamlTestSuiteIT.test {p0=upgraded_cluster/10_basic/Find a task result record from the old cluster}" -Dtests.seed=F15342E6CB740FD6 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=uk-UA -Dtests.timezone=Europe/Belfast -Druntime.java=19 -Dtests.rest.suite=upgraded_cluster
2> java.lang.AssertionError: Failure at [upgraded_cluster/10_basic:107]: hits.total didn't match expected value:
hits.total: expected Integer [1] but was Integer [0]
at __randomizedtesting.SeedInfo.seed([F15342E6CB740FD6:79077D3C6588622E]:0)
at org.opensearch.test.rest.yaml.OpenSearchClientYamlSuiteTestCase.executeSection(OpenSearchClientYamlSuiteTestCase.java:460)
at org.opensearch.test.rest.yaml.OpenSearchClientYamlSuiteTestCase.test(OpenSearchClientYamlSuiteTestCase.java:433)
at java.****/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.****/java.lang.reflect.Method.invoke(Method.java:578)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.****/java.lang.Thread.run(Thread.java:1589)
Caused by:
java.lang.AssertionError: hits.total didn't match expected value:
hits.total: expected Integer [1] but was Integer [0]
at org.opensearch.test.rest.yaml.section.MatchAssertion.doAssert(MatchAssertion.java:115)
at org.opensearch.test.rest.yaml.section.Assertion.execute(Assertion.java:89)
at org.opensearch.test.rest.yaml.OpenSearchClientYamlSuiteTestCase.executeSection(OpenSearchClientYamlSuiteTestCase.java:449)
... 39 more
2> NOTE: leaving temporary files on disk at: /var/jenkins/workspace/gradle-check/search/qa/rolling-upgrade/build/testrun/v2.8.0#upgradedClusterTest/temp/org.opensearch.upgrades.UpgradeClusterClientYamlTestSuiteIT_F15342E6CB740FD6-002
2> NOTE: test params are: codec=Asserting(Lucene95), sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=uk-UA, timezone=Europe/Belfast
2> NOTE: Linux 5.15.0-1033-aws amd64/Eclipse Adoptium 19.0.2 (64-bit)/cpus=96,threads=1,free=38070576,total=1178599424
2> NOTE: All tests run in this JVM: [IndexingIT, RecoveryIT, RefreshVersionInClusterStateIT, SystemIndicesUpgradeIT, UpgradeClusterClientYamlTestSuiteIT]
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.BootstrapForTesting (file:/var/jenkins/workspace/gradle-check/search/test/framework/build/distributions/framework-3.0.0-SNAPSHOT.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.BootstrapForTesting
WARNING: System::setSecurityManager will be removed in a future release
REPRODUCE WITH: ./gradlew ':qa:rolling-upgrade:v2.8.0#upgradedClusterTest' --tests "org.opensearch.upgrades.UpgradeClusterClientYamlTestSuiteIT.test {p0=upgraded_cluster/10_basic/Find a task result record from the old cluster}" -Dtests.seed=F15342E6CB740FD6 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=uk-UA -Dtests.timezone=Europe/Belfast -Druntime.java=19 -Dtests.rest.suite=upgraded_cluster
org.opensearch.upgrades.UpgradeClusterClientYamlTestSuiteIT > test {p0=upgraded_cluster/10_basic/Find a task result record from the old cluster} FAILED
java.lang.AssertionError: Failure at [upgraded_cluster/10_basic:107]: hits.total didn't match expected value:
hits.total: expected Integer [1] but was Integer [0]
at __randomizedtesting.SeedInfo.seed([F15342E6CB740FD6:79077D3C6588622E]:0)
at org.opensearch.test.rest.yaml.OpenSearchClientYamlSuiteTestCase.executeSection(OpenSearchClientYamlSuiteTestCase.java:460)
at org.opensearch.test.rest.yaml.OpenSearchClientYamlSuiteTestCase.test(OpenSearchClientYamlSuiteTestCase.java:433)
at java.****/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.****/java.lang.reflect.Method.invoke(Method.java:578)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.****/java.lang.Thread.run(Thread.java:1589)
Caused by:
java.lang.AssertionError: hits.total didn't match expected value:
hits.total: expected Integer [1] but was Integer [0]
at org.opensearch.test.rest.yaml.section.MatchAssertion.doAssert(MatchAssertion.java:115)
at org.opensearch.test.rest.yaml.section.Assertion.execute(Assertion.java:89)
at org.opensearch.test.rest.yaml.OpenSearchClientYamlSuiteTestCase.executeSection(OpenSearchClientYamlSuiteTestCase.java:449)
... 39 more
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.gradle.api.internal.tasks.testing.worker.TestWorker (file:/home/ubuntu/.gradle/wrapper/dists/gradle-8.1.1-all/bs1rrjki8hh9bujwbsqnxtuzr/gradle-8.1.1/lib/plugins/gradle-testing-****-8.1.1.jar)
WARNING: Please consider reporting this to the maintainers of org.gradle.api.internal.tasks.testing.worker.TestWorker
WARNING: System::setSecurityManager will be removed in a future release
