facebookresearch
Torch tensors throw 'input not a numpy array' on some classes, like faiss.Clustering
Summary
Platform
OS: Ubuntu 20.04
Faiss version: 1.7.1
Installed from: anaconda by conda install -c pytorch -c conda-forge faiss-gpu
Faiss compilation options:
Running on:
- [ ] CPU
- [x] GPU
Interface:
- [ ] C++
- [x] Python
Reproduction instructions
Example 1:
Fixed by importing
faiss.contrib.torch_utils. But that's not too obvious from wiki documentation.
import faiss.contrib.torch_utils
import faiss, torch
D = 32
bank = torch.randn(1000, D)
query = torch.randn(100, D)
res = faiss.StandardGpuResources()
idx = faiss.index_factory(D, 'IVF384,Flat')
gidx = faiss.index_cpu_to_gpu(res, 0, idx)
gidx.train(bank) # fails
gidx.train(bank.cuda()) # fails
gidx.train(bank.numpy()) # succeeds
gidx.add(bank) # fails
gidx.add(bank.cuda()) # fails
# ... etc.
Example 2:
Follows https://github.com/facebookresearch/faiss/blob/main/faiss/gpu/test/torch_test_contrib_gpu.py#L255 But fails even with
import faiss.contrib.torch_utils.
import faiss.contrib.torch_utils
import faiss, torch
D = 32
bank = torch.randn(1000, D)
query = torch.randn(100, D)
res = faiss.StandardGpuResources()
clus = faiss.Clustering(D, 384)
clus.verbose = True
clus.niter = 20
cfg = faiss.GpuIndexFlatConfig()
cfg.useFloat16 = True
cfg.device = 0
index = faiss.GpuIndexFlatIP(res, D, cfg)
clus.train(bank, index)
# Fails by
# ValueError: input not a numpy array
Torch support is enabled by an import: import faiss.contrib.torch_utils. Just put it somewhere at the top then passing torch tensors works fine.
@mdouze - Example 2 seems like a bug still. The test example doesn't work for the version installed from conda (and also tried again from pip).
Only the index classes are covered by the pytorch bridge at the moment, not Clustering or other objects. I'll add this to the feature list to implement.
