Save and load compressed index

[xudongguan202]

Hi,

I got a total 755G index saved in my disk after encoding the whole wiki passage. The large index takes huge storage and long time to load to GPU. However, it requires less than 100G after loading to GPU, which could be the index compression mentioned in your paper. Is it possible to save and load the compressed index for better time and storage consumption?

[Vincent-ch99]

Yeh, I met the same situation like u I loaded the embeddings and passages of precomputing, but when I run evaluate.py, it always shows cuda out of memory while loading. I have an A100 80G GPU memory. How much GPU memory is needed at least to load the precomputing embedding? And I also encountered the problem of extremely slowly loading. Is there any way to optimize it?

[xudongguan202]

@Vincent-ch99 At least 2x A100 80G GPUs are required to run the evaluation using the default configurations from my experiment.

The extremely slow loading could be due to index compression of saved index, as mentioned in the paper. Therefore, I am currently trying to find a way to save and load the compressed index for faster loading and light storage.