fdb-record-layer icon indicating copy to clipboard operation
fdb-record-layer copied to clipboard

Published 20 hours ago • verified publisher icon FoundationDB

LuceneIndexMaintenanceTest > flakyMerge(true, true, true, 5, -3487483610997135943, false) fails

Open ScottDugas opened this issue 10 months ago • 1 comments 

LuceneIndexMaintenanceTest > flakyMerge(boolean, boolean, boolean, int, long, boolean) > com.apple.foundationdb.record.lucene.LuceneIndexMaintenanceTest.flakyMerge(boolean, boolean, boolean, int, long, boolean)[6] FAILED
    org.opentest4j.AssertionFailedError: Group: (0), partition: 0 ==> 
expected: <[(-1, [0, 1001], [0, 1000]), (-1, [0, 1002], [0, 999]), (-1, [0, 1003], [0, 998]), (-1, [0, 1004], [0, 997]), (-1, [0, 1005], [0, 996]), (-1, [0, 1006], [0, 995]), (-1, [0, 1007], [0, 994]), (-1, [0, 1008], [0, 993]), (-1, [0, 1009], [0, 992]), (-1, [0, 1010], [0, 991]), (-1, [0, 1011], [0, 990]), (-1, [0, 1012], [0, 989]), (-1, [0, 1013], [0, 988]), (-1, [0, 1014], [0, 987])]> 
but was:  <[(-1, [0, 1001], [0, 1000]), (-1, [0, 1001], [0, 1000]), (-1, [0, 1001], [0, 1000]), (-1, [0, 1001], [0, 1000]), (-1, [0, 1001], [0, 1000]), (-1, [0, 1001], [0, 1000]), (-1, [0, 1001], [0, 1000]), (-1, [0, 1001], [0, 1000]), (-1, [0, 1001], [0, 1000]), (-1, [0, 1001], [0, 1000]), (-1, [0, 1001], [0, 1000]), (-1, [0, 1001], [0, 1000]), (-1, [0, 1001], [0, 1000]), (-1, [0, 1001], [0, 1000]), (-1, [0, 1002], [0, 999]), (-1, [0, 1002], [0, 999]), (-1, [0, 1002], [0, 999]), (-1, [0, 1002], [0, 999]), (-1, [0, 1002], [0, 999]), (-1, [0, 1002], [0, 999]), (-1, [0, 1002], [0, 999]), (-1, [0, 1002], [0, 999]), (-1, [0, 1002], [0, 999]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1003], [0, 998]), (-1, [0, 1004], [0, 997]), (-1, [0, 1004], [0, 997]), (-1, [0, 1004], [0, 997]), (-1, [0, 1004], [0, 997]), (-1, [0, 1004], [0, 997]), (-1, [0, 1005], [0, 996]), (-1, [0, 1005], [0, 996]), (-1, [0, 1005], [0, 996]), (-1, [0, 1005], [0, 996]), (-1, [0, 1006], [0, 995]), (-1, [0, 1006], [0, 995]), (-1, [0, 1006], [0, 995]), (-1, [0, 1006], [0, 995]), (-1, [0, 1006], [0, 995]), (-1, [0, 1006], [0, 995]), (-1, [0, 1006], [0, 995]), (-1, [0, 1006], [0, 995]), (-1, [0, 1006], [0, 995]), (-1, [0, 1006], [0, 995]), (-1, [0, 1006], [0, 995]), (-1, [0, 1006], [0, 995]), (-1, [0, 1006], [0, 995]), (-1, [0, 1006], [0, 995]), (-1, [0, 1006], [0, 995]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1007], [0, 994]), (-1, [0, 1008], [0, 993]), (-1, [0, 1008], [0, 993]), (-1, [0, 1008], [0, 993]), (-1, [0, 1008], [0, 993]), (-1, [0, 1008], [0, 993]), (-1, [0, 1008], [0, 993]), (-1, [0, 1008], [0, 993]), (-1, [0, 1008], [0, 993]), (-1, [0, 1008], [0, 993]), (-1, [0, 1008], [0, 993]), (-1, [0, 1008], [0, 993]), (-1, [0, 1008], [0, 993]), (-1, [0, 1008], [0, 993]), (-1, [0, 1008], [0, 993]), (-1, [0, 1008], [0, 993]), (-1, [0, 1008], [0, 993]), (-1, [0, 1009], [0, 992]), (-1, [0, 1009], [0, 992]), (-1, [0, 1009], [0, 992]), (-1, [0, 1009], [0, 992]), (-1, [0, 1009], [0, 992]), (-1, [0, 1009], [0, 992]), (-1, [0, 1009], [0, 992]), (-1, [0, 1009], [0, 992]), (-1, [0, 1010], [0, 991]), (-1, [0, 1010], [0, 991]), (-1, [0, 1010], [0, 991]), (-1, [0, 1010], [0, 991]), (-1, [0, 1010], [0, 991]), (-1, [0, 1010], [0, 991]), (-1, [0, 1010], [0, 991]), (-1, [0, 1011], [0, 990]), (-1, [0, 1011], [0, 990]), (-1, [0, 1011], [0, 990]), (-1, [0, 1011], [0, 990]), (-1, [0, 1011], [0, 990]), (-1, [0, 1011], [0, 990]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1012], [0, 989]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1013], [0, 988]), (-1, [0, 1014], [0, 987]), (-1, [0, 1014], [0, 987]), (-1, [0, 1014], [0, 987]), (-1, [0, 1014], [0, 987]), (-1, [0, 1014], [0, 987]), (-1, [0, 1014], [0, 987]), (-1, [0, 1014], [0, 987]), (-1, [0, 1014], [0, 987]), (-1, [0, 1014], [0, 987]), (-1, [0, 1014], [0, 987]), (-1, [0, 1014], [0, 987]), (-1, [0, 1014], [0, 987]), (-1, [0, 1014], [0, 987]), (-1, [0, 1014], [0, 987])]>
        at app//org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
        at app//org.junit.jupiter.api.AssertionUtils.failNotEqual(AssertionUtils.java:62)
        at app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
        at app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1135)
        at app//com.apple.foundationdb.record.lucene.LuceneIndexTestValidator.validatePrimaryKeySegmentIndex(LuceneIndexTestValidator.java:300)
        at app//com.apple.foundationdb.record.lucene.LuceneIndexTestValidator.validate(LuceneIndexTestValidator.java:199)
        at app//com.apple.foundationdb.record.lucene.LuceneIndexMaintenanceTest.flakyMerge(LuceneIndexMaintenanceTest.java:392)

ScottDugas avatar Apr 22 '24 13:04 ScottDugas

I suspect it is either:

  1. We are incorrect in our assumption that a successful merge should always cleanup the files
  2. There is a bug that is causing the file deletion to not cleanup the primaryKeySegmentIndex properly.

(1) may only result in a temporary growth in the primaryKeySegmentIndex, and a temporary slowdown when deleting documents. (2) on the other hand could result in the primaryKeySegmentIndex to grow unbounded.

ScottDugas avatar Apr 22 '24 13:04 ScottDugas