Jeff Johnson

icon indicating a website link https://www.linkedin.com/in/jeff-johnson-698a822/

icon company Meta Fundamental AI Research (FAIR) icon location Jackson, WY icon location SIMD + GPU + FPGA + ASIC stuff for AI/ML. I wrote the original PyTorch GPU backend, GPU Faiss, and many other AI GPU things broadly in use across the industry.

Results 9 comments of Jeff Johnson

faiss::gpu::runMatrixMult ... cublas failed (13): (1024, 12) x (256, 12)' = (1024, 256) gemm params m 256 n 1024 k 12 trA T trB N lda 12 ldb 12 ldc 256

What kind of GPU are you using? 40 GiB makes me think of A100, which really should require CUDA 11?

cudaMalloc error out of memor Error

This is due to the geometric doubling behavior in faiss::gpu::DeviceVector's append, which happens when you call add on an index which already has data in it: https://github.com/facebookresearch/faiss/blob/main/faiss/gpu/utils/DeviceVector.cuh#L94 This is something...

range_search on GPU

This wouldn't be easy to implement, and I don't have the time to do it, but may be a long term feature to add.

range_search on GPU

> def radius_search(index, x, thresh, nprobe, max_k=3200, init_k=100, gpu=False): note, the maximum k-select value on the GPU is 2048 at present

GPU memory usage increases after creating new instances in different threads

Any GPU indices that share a single GpuResources object can only be used one at a time from any thread. A single GpuResources object can only be operated on sequentially,...

Faiss GPU: improve error information for GPU OOM

@cjnolet FYI with the Faiss/RAFT integration when the RAFT memory manager was being used, we were still reserving 1.5 GB of memory taken from the RAFT allocator which was given...

Faiss GPU: improve error information for GPU OOM

@cjnolet What is your thought on this change to the memory allocator? Currently in the repo, it was using (likely by accident) *both* the Faiss and RAFT memory allocator. The...

Creating an array of faiss.Kmeans objects that uses gpu

You seem to be passing a pytorch GPU tensor as input? You need to import faiss.contrib.torch_utils in order to allow this.

faiss.contrib.torch_utils: torch.TypedStorage (and torch.Tensor.storage) are deprecated with pytorch 2

This should just be undoing https://github.com/facebookresearch/faiss/pull/2631 except instead of `_storage()` we use `untyped_storage()` https://pytorch.org/docs/stable/generated/torch.Tensor.untyped_storage.html#torch.Tensor.untyped_storage however, this means that this code needs to differ between pytorch < 2 and pytorch >...