faiss-cpu uses GPU

[frankandrobot]

System Info

Langchain version: latest environment: latest Google collab

Who can help?

@eyurtsev

Information

• [ ] The official example notebooks/scripts
• [X] My own modified scripts

Related Components

• [ ] LLMs/Chat Models
• [ ] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [X] Vector Stores / Retrievers
• [x] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [ ] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

You can see the issue here: https://colab.research.google.com/drive/1WaW5WAHTN7xlfo_jK9AnnpjfvVSQITqe?usp=sharing

Step 0. Install faiss-cpu Step 1. Load a large model that takes up most of the GPU memory Step 2. Call FAISS.from_documents Step 3. Error out with OutOfMemoryError: CUDA out of memory.

Expected behavior

• We're using faiss-cpu, so the expectation is that all FAISS ops are on the CPU
• However, we get GPU OutOfMemoryErrors when trying to shove docs into the vector store, so 🤷