tantivy
tantivy copied to clipboard
quickwit-oss
add search_tokenizer on TextFieldIndexing
This feature let's you define a different tokenizer for searching than indexing. This doesn't break previous functionality, because it defaults to the indexing tokenizer if search_tokenizer was not specified.
This is called "search_analyzer" in Elasticsearch. https://www.elastic.co/guide/en/elasticsearch/reference/current/search-analyzer.html
That link also explains why it's useful:
Sometimes, though, it can make sense to use a different analyzer at search time, such as when using the edge_ngram tokenizer for autocomplete or when using search-time synonyms.
Issue for this PR: #1074
@mlvzk Can you open an issue to host the discussion? (In the future, please do that before opening a PR.)
@fulmicoton I opened an issue. Was that what you meant? I couldn't find this in other PRs, other than your last one. Should I just open a PR with Issue/<issue_id> as title next time?
@mlvzk Yes it is fine.
ideally, it is better to add Closes #1074 in your commit message to automatically link the PR and the issue, and to autoclose the issue upon closing.
We don't do that for all PR though.
@mlvzk what is the status of this PR? Have you addressed the changes from @PSeitz ?