autofaiss
autofaiss copied to clipboard
use feature to specify the scanner in an ivf index to implement one stage filtering
https://github.com/facebookresearch/faiss/wiki/Inverted-list-objects-and-scanners#the-invertedlistscanner-object
this would require some c++ code, but it seems possible to have one stage filtering directly with faiss by overriding that scanner to add a masking feature (for example mask to keep only samples of a given language, or a given partner)