rucene icon indicating copy to clipboard operation
rucene copied to clipboard

Does Rucene support Chinese character indexing and searching

Open qiqi-arch opened this issue 4 years ago • 1 comments

This might be a silly question, does Rucene support Chinese character indexing and searching.

I don't see any tokenizer under the https://github.com/zhihu/rucene/tree/master/src/core/analysis

qiqi-arch avatar Mar 17 '20 16:03 qiqi-arch

This might be a silly question, does Rucene support Chinese character indexing and searching.

I don't see any tokenizer under the https://github.com/zhihu/rucene/tree/master/src/core/analysis

Hi Qiqi, sorry for the delay. And the quick answer is that we don't have tokenizer in rucene at this time. We resort to data pipeline to tokenize the input instead.

sunxiaoguang avatar Apr 02 '20 06:04 sunxiaoguang