Facilitating usage as a vector codec

Open mdouze opened this issue 1 year ago • 5 comments

For some use cases, Faiss serves only as a storage engine for flat arrays of compressed vectors. Maybe it would be useful to make a wrapper that makes this easier to use (especially to make compression transparent). More details in this doc (visible internally in Meta):

https://docs.google.com/document/d/1v1KkD4KqrgHyjqgfykRQss37LdozZhqPMTYbymJlvDw/edit?usp=sharing

Oct 19 '24 09:10 mdouze

@mdouze would it be possible to see this document? Thanks

Oct 21 '24 15:10 alexanderguzhva

@mdouze would it be possible to see this document? Thanks

Jun 09 '25 07:06 navneet1v

Please ask for access, we will share.

Jun 09 '25 16:06 mdouze

Please ask for access, we will share.

done.

Jun 09 '25 19:06 navneet1v

I did end up building a prototype of this library – please try it out, fork, etc!

https://github.com/jxmorris12/embzip

Jun 10 '25 17:06 jxmorris12