fdb-record-layer

fdb-record-layer copied to clipboard

The current system sequentially opens the readers. `com.apple.foundationdb.record.lucene.directory.FDBIndexInput.length(FDBIndexInput.java:197) org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:513) org.apache.lucene.codecs.CodecUtil.retrieveChecksum(CodecUtil.java:490) org.apache.lucene.codecs.CodecUtil.retrieveChecksum(CodecUtil.java:509) org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.(BlockTreeTermsReader.java:240) com.apple.foundationdb.record.lucene.codec.LuceneOptimizedPostingsFormat.fieldsProducer(LuceneOptimizedPostingsFormat.java:57) org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:114) org.apache.lucene.index.SegmentReader.(SegmentReader.java:83) org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:69) org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:61) org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:720) org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:84) org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:64) com.apple.foundationdb.record.lucene.directory.FDBDirectoryWrapper.getReader(FDBDirectoryWrapper.java:85) com.apple.foundationdb.record.lucene.directory.FDBDirectoryManager.getIndexReader(FDBDirectoryManager.java:119) com.apple.foundationdb.record.lucene.LuceneRecordCursor.getIndexReader(LuceneRecordCursor.java:292) com.apple.foundationdb.record.lucene.LuceneRecordCursor.performScan(LuceneRecordCursor.java:297)`

jleach4

Resolves #720 Auto-throttling runner

49

This introduces a new `LimitedRunner` and `TransactionalLimitedRunner` which replace the old `IndexingThrottle` in a generic way. There are some properties of `FDBDatabaseRunner` that used to be changeable in the middle...

ScottDugas

Current State is rough

2

jleach4

Today we have the `FDBDatabaseRunner` which can be used to iteratively perform work and to repeatedly run a transaction to success, however for certain tasks the success or failure of...

scgray

Transition SortedDocValuesField to BinaryPoint to support efficient deletes in Lucene #1595

7

The SortedDocValuesField is known to have issues deleting large number of records and requires the index to read more data than needed during a delete. The BinaryPoint implementation is known...

jleach4

LuceneIndexTest.simpleInsertDeleteAndSearch is flaky, I've seen it fail a few times with no relevant changes in https://github.com/FoundationDB/fdb-record-layer/pull/1660 with: ``` LuceneIndexTest > simpleInsertDeleteAndSearch() FAILED org.opentest4j.AssertionFailedError: expected: but was: at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55) at org.junit.jupiter.api.AssertTrue.assertTrue(AssertTrue.java:40)...

ScottDugas

Right now, `planOr` is attempted before any indexes, even though in the case of Lucene `planOther` can handle the entire disjunction. Since the planner only currently matches Lucene indexes based...

MMcM

tian-yizuo

It actually only needs to save primary key fields that aren't in the grouping key. I think it's pretty common for these to overlap. Unfortunately, changing this will be incompatible...

MMcM

At this point, we've been publishing dual proto2 and proto3 versions of the Record Layer for a while. This complicates our builds in a number of ways. We should be...

alecgrieser