data-prepper
data-prepper copied to clipboard
[BUG] Flaky tests high-level tracking
Describe the bug Running the data-prepper build and github actions frequently has failing flaky tests, and often requires retrying the tests multiple times before it succeeds. This issue is meant to allow easy tracking of these flaky tests, by adding an item to the checklist when a flaky test is seen.
Flaky tests
- [ ]
> Task :data-prepper-core:integrationTest PipelinesWithAcksIT > simple_pipeline_with_multiple_records() FAILED java.lang.NullPointerException at PipelinesWithAcksIT.java:85
- [ ]
testcase name="three_pipelines_with_route_and_multiple_records()" classname="org.opensearch.dataprepper.integration.PipelinesWithAcksIT" time="11.003"> <failure message="java.lang.NullPointerException" type="java.lang.NullPointerException">java.lang.NullPointerException at org.opensearch.dataprepper.integration.PipelinesWithAcksIT.three_pipelines_with_route_and_multiple_records(PipelinesWithAcksIT.java:130)
- [x] #3470
- [ ] #1374
- [ ] Router_ThreeRoutesIT https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-1793501480
- [ ] AwsCloudMapPeerListProviderTest.java:345 - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-1795279643
- [ ] OTelTraceRawProcessorTest.java:270 - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-1796339462
- [ ] OTelTraceRawProcessorTest.java:193 - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-1806598375
- [x] #4298
- [ ] HistogramAggregateActionTests.java:370 - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-2207244965
- [x] LambdaSinkServiceTest.java:316 - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-2221378060; fixed by #4723
- [ ] RDS ExportSchedulerTest - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-2393877779
- [ ] RDS StreamScheduler - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-2394058183. (added sleep time in #5016, will check again see if this helps)
- [ ] Truncate processor - https://github.com/opensearch-project/data-prepper/issues/3481#issuecomment-2405626969