flow-framework
flow-framework copied to clipboard
[BUG] Reprovision Workflow IT is flaky
What is the bug?
FlowFrameworkRestApiIT.testReprovisionWorkflow() is flaky. During 2.17.0 release it required multiple retries to pass on Windows x86. There are occasional GitHub CI failures on other platforms, see https://github.com/opensearch-project/flow-framework/actions/runs/10777769278/job/29887668921#step:4:59 (Ubuntu) and https://github.com/opensearch-project/flow-framework/actions/runs/10777865339/job/29888745661#step:4:49 (macOS)
FlowFrameworkRestApiIT > testReprovisionWorkflow FAILED
org.junit.ComparisonFailure: expected:<[COMPLET]ED> but was:<[FAIL]ED>
at __randomizedtesting.SeedInfo.seed([4D7D6[49](https://github.com/opensearch-project/flow-framework/actions/runs/10777865339/job/29888745661#step:4:50)945752BD3:196DC17D8E477B7D]:0)
at org.junit.Assert.assertEquals(Assert.java:117)
at org.junit.Assert.assertEquals(Assert.java:146)
at org.opensearch.flowframework.FlowFrameworkRestTestCase.getAndAssertWorkflowStatus(FlowFrameworkRestTestCase.java:7[53](https://github.com/opensearch-project/flow-framework/actions/runs/10777865339/job/29888745661#step:4:54))
at org.opensearch.flowframework.FlowFrameworkRestTestCase.lambda$getResourcesCreated$7(FlowFrameworkRestTestCase.java:805)
at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1136)
at org.opensearch.flowframework.FlowFrameworkRestTestCase.getResourcesCreated(FlowFrameworkRestTestCase.java:804)
at org.opensearch.flowframework.rest.FlowFrameworkRestApiIT.testReprovisionWorkflow(FlowFrameworkRestApiIT.java:471)
How can one reproduce the bug?
Run our CI repeatedly. It fails reasonably often.
What is the expected behavior?
Tests consistently pass.
What is your host/environment?
Reproducible on Windows, Linux, macOS as shown above
Do you have any additional context?
The failure is occurring consistently on the attempt to reprovision to remove default ingest pipeline (setting it to _none, which is the SearchPipelineService.NOOP_PIPELINE_ID).
The change in default pipeline name is the only difference in the two templates.