sqlite-vec icon indicating copy to clipboard operation
sqlite-vec copied to clipboard

KD-Tree or Ball-Tree Algorithms for kNN search?

Open RayHackett opened this issue 1 year ago • 2 comments

Thanks for developing the library. So far using to run a KNN on embedding vectors it has been pretty straight forward.
Are there any plans to speed up KNN search by introducing clever algorithms like a K-D tree or Ball-Tree? Running a brute force search on every query seems a bit wasteful for large data tables.
It would be amazing if create index ... ON Table(vector) was actually creating a K-D tree.

Best wishes!

RayHackett avatar Oct 02 '24 14:10 RayHackett

Is this the same as #25? (I am new here, sorry if irrelevant!)

mholt avatar Oct 08 '24 12:10 mholt

I suppose that depends on how you implement k-d or ball tree algorithms though they could be.
There seem to be some ANN implementations for SQLite. I was looking at this ticket #94 for alternatives.
This ticket was more aimed at faster exact kNN search algorithms before trying out approximate NNs. I realize implementing any of this is not at all trivial.

RayHackett avatar Oct 08 '24 13:10 RayHackett