Paul Masurel comments

Results 327 comments of


                                            Paul Masurel

Supporting tokenizer register

I don't have time to work on this but any help is welcome.

Add index size as benchmark

That sounds like a good idea.

Add index size as benchmark

@usamec Correct. It would be nice to have that in the benchmark though.

Random Crash in Bitpacking/Columnar when Merging Segments

The code you point at is NOT using SIMD. SIMD bitpacking uses a different layout that prevents efficient random access, and we need random access for columns. Like @PSeitz, I...

Avoid converting BTreeMap to Vec in TantivyDocument::add_object.

@adamreichold it won't compile.

Avoid converting BTreeMap to Vec in TantivyDocument::add_object.

I don't quite understand the motivation for this PR. @adamreichold ?

split Term into Term and IndexingTerm

As discussed offline, this looks good but let's not merge not progress on #1795 at the moment.

Speed up searches by removing repeated memsets coming from vec.resize()

I think there is a problem in the benchmark. That code is not used in most of the queries showing an improvements.

Any plan to support learned sparse vector search?

@cyccbxhl no plans at the moment. It is worth to leave the issue open. Several companies in that space are using tantivy. This ticket could spark a discussion.

feat: cardinality aggregation

@PSeitz I leave you the rest of the code review. Also can you open a ticket to one or another isolate aggregation? I think it is a bit overkill for...