Rohan Chitale
Rohan Chitale
@lowener I've updated the issue with the recall I get when I use FAISS. Strangely, the recall for `coherev2-dbpedia` is higher for cuVS compared to FAISS. I also consistently see...
Thanks @lowener. For reference, I used `cuvs` version 24.12 and I built `faiss` off of this commit: https://github.com/facebookresearch/faiss/commit/df6a8f6b4e6ed4c509e52d1e015f87fd742c17df
Interestingly, I found that using `ivf_pq_build_params` instead of `ivf_pq_params` here: https://github.com/navneet1v/VectorSearchForge/blob/main/cuvs_benchmarks/main.py#L375 resulted in higher recall and faster index build times for the `marco-tasb` and `cohere-768-ip` datasets. I saw no difference...
Thank you @lowener! Will this code change also improve the recall we see with faiss for `cohere-768-ip`? The 82.6% recall we see with faiss is still lower than the recall...
Hi @lowener, I built faiss from source using cuvs 25.06. I verified the recall is now 91.9% for the cohere-768 IP dataset using faiss-cuvs - this is a big improvement...
Here are my new faiss-cuvs and cuvs results, with cuvs-25.06: | Dataset name | Location | Space type | Dimensions | Documents | Normalized | Recall with cuVS API |...
@tfeher Here is the results of the `rmm` logs, with using `mmap` to load the vectors: [rmm_log_mmap.csv](https://github.com/user-attachments/files/19915410/rmm_log_mmap.csv) Here is the results of the `rmm` logs, without using `mmap` to load...
@tfeher Graph with using mmap:  Graph without using mmap:  The memory usage stats don't match what I see when i use `nvidia-smi`; these graphs look exactly the same....
Please note that we are open for feedback on this, as this feature is still a work in progress.
To add on - the worker calculates the resource requirements for the job up front, before deciding whether to accept or reject it. We use a pretty naive approach, that's...