tinysearch icon indicating copy to clipboard operation
tinysearch copied to clipboard

Benchmark against competition

Open mre opened this issue 5 years ago • 7 comments

People are interested in a size-comparison between

  • tinysearch
  • lunr.js
  • elasticlunr
  • flexsearch
  • fuse.js

If anyone wants to do a comparison, post a comment here. Some ideas:

  • Use an open dataset, e.g. Shakespeare texts https://shakespeare.folger.edu/download/
  • Create a size histogram of the results that we can include into the README.
  • Add the code to the repository under a bench folder and create a pull request. This will make the benchmark reproducible in the future.

Feel free to ask questions here before starting.

mre avatar Oct 13 '20 11:10 mre

Awesome idea. I use Zola for my blog which comes with elasticlunr. I am evaluating different search solutions.

Might want to add Stork to this list as well.

Jieiku avatar May 01 '22 21:05 Jieiku

I found that somebody actually made one that tests a lot of them!

https://nextapps-de.github.io/flexsearch/bench/

Unfortunately tinysearch is not in their benchmark list.

(neither is pagefind)

Jieiku avatar Apr 13 '23 04:04 Jieiku

Interesting. We should add it there; also Stork and pagefind.

mre avatar Apr 13 '23 08:04 mre

one thing I dont see in their benchmark is bandwidth used... so they really should have a second benchmark for that.

(that is just as important to me as performance, if they are not interested in making one then I might.)

Jieiku avatar Apr 13 '23 17:04 Jieiku

How exactly do you define bandwidth? Queries per second? You could create an issue for the project to discuss that.

mre avatar Apr 13 '23 22:04 mre

no, what i mean is hit the website. perform a search, how many KB had to be downloaded for the search javascript + wasm + index to perform that search.

For instance Stork is around 700kb whereas tinysearch is around 100kb for abridge.

Obviously the amount of content will affect the index size. In their test they appear to be using Gulliver's Travels

they could also benchmark say: 1search, 10searches, etc. to see if that affects the amount of data downloaded to perform the searches, obviously this would affect pagesearch and possibly others because they chunk the index into multiple pieces.

Jieiku avatar Apr 14 '23 00:04 Jieiku

Gotcha. Yeah, that would indeed be a great comparison.

mre avatar Apr 14 '23 09:04 mre