elasticlunr-rs
elasticlunr-rs copied to clipboard
mattico
A partial port of elasticlunr to Rust. Intended to be used for generating compatible search indices.
This gets rid of the extremely slow regex based implementation & introduces an all new implementation taken from [rust-stem](https://github.com/minhnhdo/rust-stem). I had to make a few changes to make it compatible...
Newer versions of lindera have a different API with some sophisticated analysis capabilities. https://github.com/lindera-morphology/lindera#analysis-example They look like they can do everything required of the search pipeline in an integrated way....
If I add a very long word to the index (8000 characters is enough on my machine), I get a stack overflow while serialising it. Here's how I was testing:...
Not included in the crate of course. elasticlunr.js is unmaintained, and it supports an old version of lunr-languages. We can do full integration testing and actually show what javascript code...
https://github.com/MihaiValentin/lunr-languages/blob/master/lunr.multi.js
The current InvertedIndex and DocumentStore implementations are direct ports of the javascript ones. Rather than stick with a nested HashMap, it would be more efficient and more elegant to use...
https://github.com/mattico/elasticlunr-rs/blob/29d97e4c8e91bb0d1813716fb2d1575066344d76/src/inverted_index.rs#L40-L42 During index building, `elasticlunr-rs` iterates over the token `&str`'s content in [Unicode Scalar Values](https://doc.rust-lang.org/std/primitive.str.html#method.chars). While the JS library does it in this way: ```js elasticlunr.InvertedIndex.prototype.addToken = function (token, tokenInfo,...
