tantivy
tantivy copied to clipboard
feat: make `BlockSegmentPostings::open` and `TermInfoStore` public
Databend uses tantivy to implement inverted index, and tantivy Searcher
needs to read all the index file data when starting up, which is very large, resulting in poor query performance. To improve performance, we implemented the Searcher
ourselves to query the matched docs, and improve performance by reading only the FST
data, TermInfo
data, and the matched terms related data to reduce the size of data to be read. We need to use BlockSegmentPostings
and TermInfoStore
, as well as some fields from the Query
.