Paul Masurel

That's an interesting idea.

That could be interesting. If someone wants to work on this, contributions are welcome.

Here is what I'd love to have for shingles to improve phrase query speed incredibly at a very low cost on indexing. A shinglefilter that takes a trigger dictionary... At...

@mocobeta For this to be useful, we need to have some asymetry between the indexing tokenizer and the search tokenizer isn't it? We need to land #1073 or some variant...

@mocobeta ah yes, you are right. We don't need any asymetry between search and indexing here. It is only helpful with the extension I suggested. LGTM. I just added a...

``` Total indexed docs: 47187784 Total term freq in unigram field: 9171726767 (unique terms: 29185) Total term freq in bigram field: 9124538983 (unique terms: 96691414) Total term freq in trigram...

I do not know which flavor is the best... I don't think this is a one size fits all problem, so differnt user will ask for different things.

Sweet ! Looking forward to read your progress @mocobeta!

Thanks for reporting. Tantivy uses block_on in some places internally... That's probably not great if some one is already running in an executor. 👍 I'll investigate different solutions.

We actually one weird use case for using tantivy within a block_on call at quickwit. I'll just remove all call to `block_on` in tantivy.