LintDB icon indicating copy to clipboard operation
LintDB copied to clipboard

Add HSNW-IVF index options

Open mtbarta opened this issue 1 year ago • 0 comments

My system is currently under pressure while training a new model. Benchmarking the search critical path, it's the coarse quantizer that takes the most time at a whopping 80%.

graph

HSNW-IVF will use HSNW to find the closest centroids and cut down on the latency.


Note: This work depends on removing the fully constructed distance matrix used in KnnNearestNeighbors.

Context: KnnNearestNeighbors helps PLAID by containing a knn distance matrix with all of the centroids. PLAID relies on this to take each document's code and lookup that code's distance.

We can always use a sparse matrix for this instead, or perhaps XTR's imputing missing distances method would work well.

mtbarta avatar Aug 20 '24 21:08 mtbarta