Caleb Hattingh
Caleb Hattingh
I will look at this within the next two weeks or so.
For (my own) future reference the upstream tantivy docs for custom tokenizers is [here](https://docs.rs/tantivy/latest/tantivy/tokenizer/index.html#custom-tokenizers).
I've started working on this in a branch here (currently incomplete): https://github.com/cjrh/tantivy-py/tree/custom-tokenizer-support I think it will be possible to add support via features as suggested. We could also consider making...
I've done a bit more work and put up my PR in draft mode #200 . I will try to add tantivy-jieba in a similar way under fflag in the...
It's a good idea, we should do this.
fyi @wallies - since we're planning on a new release soon anyway maybe now is a good time to do this.
Had a look. Sadly, `PyDateTime` for now is [not supported](https://docs.rs/pyo3/0.19.2/pyo3/types/struct.PyDateTime.html#) in the limited ABI and we [use it](https://github.com/quickwit-oss/tantivy-py/blob/master/src/document.rs#L9) in `document.rs`.
Apparently there is some kind of fallback support for datetime described in this PR: https://github.com/PyO3/pyo3/issues/3633. I need to look at it but perhaps there's a path for us.
Field boosting was recently added in this pull request: https://github.com/quickwit-oss/tantivy-py/pull/202 It has not yet been added to the documentation.
@tem-code Please don't be shy to make a PR to fix he types file 😉