faiss-rs icon indicating copy to clipboard operation
faiss-rs copied to clipboard

Write Index into memory or Read Index from memory

Open ava57r opened this issue 2 years ago • 1 comments

Hello.

C++ Faiss supports some methods for saving a index into memory.

write into IOWriter https://github.com/facebookresearch/faiss/blob/main/faiss/index_io.h#L40 and impl VectorIOWriter https://github.com/facebookresearch/faiss/blob/main/faiss/impl/io.h#L59

read from IOReader https://github.com/facebookresearch/faiss/blob/main/faiss/index_io.h#L61 and impl VectorIOReader https://github.com/facebookresearch/faiss/blob/main/faiss/impl/io.h#L53

Could We add these impls into rust library?

ava57r avatar Nov 08 '22 17:11 ava57r

We would be well served with standard Rust APIs for writers and readers, so that an index can be written to anything implementing std::io::Write and an index can be read from anything implementing std::io::Read. I am not fully familiar with how these data structures operate, but so long as they can work with preallocated input/output buffers, it should be possible to interoperate them.

Enet4 avatar Nov 29 '22 18:11 Enet4