neural-search icon indicating copy to clipboard operation
neural-search copied to clipboard

[BUG] Flaky test: testReduceWithConcurrentSegmentSearch_whenMultipleCollectorsMatchedDocs_thenSuccessful

Open chishui opened this issue 7 months ago • 2 comments

What is the bug?

testReduceWithConcurrentSegmentSearch_whenMultipleCollectorsMatchedDocs_thenSuccessful failed in gradle check: https://github.com/opensearch-project/neural-search/actions/runs/9788202904/job/27025892009?pr=769

org.opensearch.neuralsearch.search.query.HybridCollectorManagerTests > testReduceWithConcurrentSegmentSearch_whenMultipleCollectorsMatchedDocs_thenSuccessful FAILED
    java.lang.IllegalArgumentException: cannot merge top docs because it does not have enough elements
        at __randomizedtesting.SeedInfo.seed([8305B866B5D072CE:EAE289E17B23685E]:0)
        at org.opensearch.neuralsearch.search.query.HybridQueryScoreDocsMerger.merge(HybridQueryScoreDocsMerger.java:39)
        at org.opensearch.neuralsearch.search.query.TopDocsMerger.merge(TopDocsMerger.java:52)
        at org.opensearch.neuralsearch.search.query.HybridCollectorManager.reduceCollectorResults(HybridCollectorManager.java:241)
        at org.opensearch.neuralsearch.search.query.HybridCollectorManager.lambda$reduce$0(HybridCollectorManager.java:135)
        at org.opensearch.neuralsearch.search.query.HybridCollectorManager.lambda$reduceSearchResults$7(HybridCollectorManager.java:254)
        at org.opensearch.neuralsearch.search.query.HybridCollectorManagerTests.testReduceWithConcurrentSegmentSearch_whenMultipleCollectorsMatchedDocs_thenSuccessful(HybridCollectorManagerTests.java:412)
REPRODUCE WITH: ./gradlew ':test' --tests "org.opensearch.neuralsearch.search.query.HybridCollectorManagerTests.testReduceWithConcurrentSegmentSearch_whenMultipleCollectorsMatchedDocs_thenSuccessful" -Dtests.seed=8305B866B5D072CE -Dtests.security.manager=false -Dtests.locale=af-ZA -Dtests.timezone=Europe/Uzhgorod -Druntime.java=11

How can one reproduce the bug?

It can be reproduced locally with command

./gradlew ':test' --tests "org.opensearch.neuralsearch.search.query.HybridCollectorManagerTests.testReduceWithConcurrentSegmentSearch_whenMultipleCollectorsMatchedDocs_thenSuccessful" -Dtests.seed=8305B866B5D072CE -Dtests.security.manager=false -Dtests.locale=af-ZA -Dtests.timezone=Europe/Uzhgorod

However, if remove the -Dtests.seed=8305B866B5D072CE, the test will pass.

What is the expected behavior?

The UT will pass with or without seed.

What is your host/environment?

linux/macos

Do you have any screenshots?

N/A

Do you have any additional context?

N/A

chishui avatar Jul 04 '24 06:07 chishui