langchain icon indicating copy to clipboard operation
langchain copied to clipboard

More choices embedding search

Open AlexYoung757 opened this issue 2 years ago • 0 comments

Feature Description An optional configuration is provided, which allows users to freely choose to use faiss local library(or annoy,hnswlib, jina) remote/distributed vector database as vector storage.

Problem Solved When the amount of data in the knowledge base is large (txt file larger than 200m), the io speed of faiss-cpu is too slow to use gpu and cannot be expanded horizontally.

Implementation Suggestions Implement an interface, which vector database can be configured by configuration file, such as annoy,hnswlib,faiss or jina.

Alternative Solutions There is no alternative plan at present

Additional Information Add any other information related to the feature request.

AlexYoung757 avatar May 17 '23 06:05 AlexYoung757