fdb-record-layer icon indicating copy to clipboard operation
fdb-record-layer copied to clipboard

Parallelize SegmentReader Creation During IndexReader.open

Open jleach4 opened this issue 2 years ago • 0 comments

The current system sequentially opens the readers.

com.apple.foundationdb.record.lucene.directory.FDBIndexInput.length(FDBIndexInput.java:197) org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:513) org.apache.lucene.codecs.CodecUtil.retrieveChecksum(CodecUtil.java:490) org.apache.lucene.codecs.CodecUtil.retrieveChecksum(CodecUtil.java:509) org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.<init>(BlockTreeTermsReader.java:240) com.apple.foundationdb.record.lucene.codec.LuceneOptimizedPostingsFormat.fieldsProducer(LuceneOptimizedPostingsFormat.java:57) org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:114) org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:83) org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:69) org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:61) org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:720) org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:84) org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:64) com.apple.foundationdb.record.lucene.directory.FDBDirectoryWrapper.getReader(FDBDirectoryWrapper.java:85) com.apple.foundationdb.record.lucene.directory.FDBDirectoryManager.getIndexReader(FDBDirectoryManager.java:119) com.apple.foundationdb.record.lucene.LuceneRecordCursor.getIndexReader(LuceneRecordCursor.java:292) com.apple.foundationdb.record.lucene.LuceneRecordCursor.performScan(LuceneRecordCursor.java:297)

jleach4 avatar May 19 '22 23:05 jleach4