thinker-fts icon indicating copy to clipboard operation
thinker-fts copied to clipboard

Using a persistent index

Open nevf opened this issue 8 years ago • 1 comments

You mention:

Allows saving/loading the index to/from disk, but for small datasets you can feed the index on-the-fly.

however I can't see any documentation about this. I'd like to store the index in a database (MongoDB) and query that.

FYI I'm looking at and evaluating the various full-text search libraries available for Node.js and the Browser and have only just found thinker-fts and from reading the documentation is looks quite good.

It would be nice to see a better online demo though. Fuse.js is excellent in this regard, but you don't need to go quite that far.

nevf avatar Nov 24 '16 07:11 nevf

By feeding the index on-the-fly, i mean running thinker.feed(...) with the full dataset like in the first example of the readme.

As the "feeding" is quite fast, saving a pre-calculated index to/from disk is only necessary for large datasets (~ >1000 full text documents) where it starts taking seconds, or for huge datasets minutes.

thinker.feed([
    [1, 'Lorem', 'Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.'],
    [2, 'Ipsum', 'Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.']
]);

I think, you can run it one entry at a time too.

result.forEach( (row) => { thinker.feed([row]) } );

I will look into making a good online demo when i find some spare hours :)

All feedback is appreciated, keep it coming :+1:

Hexagon avatar Nov 24 '16 22:11 Hexagon