tantivy icon indicating copy to clipboard operation
tantivy copied to clipboard

feat: make `BlockSegmentPostings::open` and `TermInfoStore` public

Open b41sh opened this issue 4 months ago • 0 comments

Databend uses tantivy to implement inverted index, and tantivy Searcher needs to read all the index file data when starting up, which is very large, resulting in poor query performance. To improve performance, we implemented the Searcher ourselves to query the matched docs, and improve performance by reading only the FST data, TermInfo data, and the matched terms related data to reduce the size of data to be read. We need to use BlockSegmentPostings and TermInfoStore, as well as some fields from the Query.

b41sh avatar Oct 16 '24 14:10 b41sh