mmap=true

[celtiberi]

Need to be able to set mmap=true for torch.load() for us humble people without very good computers.

That is all.

For those that need a Fix now just make your own ColPaliModel

class CustomColPaliModel(ColPaliModel): ... def from_index( ... instance._load_index_with_mmap(index_path) return instance } ... def _load_index_with_mmap(self, index_path: Path): ... loaded_tensor = torch.load(file, map_location=map_location, mmap=True) ...

If you are using Apple Silicon do NOT use 'mps' when loading the index. Only 'cpu'. mmap does not work otherwise.